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1 Introduction 

NASA is engaged in a major research effort towards the development of a practical validation 
and verification methodology for digital fly-by-wire control systems. 1 Researchers at NASA 
Langley Research Center (LaRC) are exploring formal verification as a candidate technology 
for the elimination of design errors in such systems. In previous reports [1, 2, 3], we put 
forward a high level architecture for a reliable computing platform (RCP) based on fault- 
tolerant computing principles. Central to this work is the use of formal methods for the 
verification of a fault-tolerant operating system that schedules and executes the application 
tasks of a digital flight control system. Phase 1 of this effort established results about the 
high level design of RCP. This report presents our Phase 2 results, which carry the design, 
specification, and verification of RCP to lower levels of abstraction. 

The major goal of this work is to produce a verified real-time computing platform, both 
hardware and operating system software, which is useful for a wide variety of control-system 
applications. Toward this goal, the operating system provides a user interface that “hides” 
the implementation details of the system such as the redundant processors, voting, clock 
synchronization, etc. We adopt a very abstract model of real-time computation, introduce 
three levels of decomposition of the model towards a physical realization, and rigorously 
prove that the decomposition correctly implements the model. Specifications and proofs 
have been mechanized using the Ehdm verification system [4]. 

A major goal of the RCP design is to enable the system to recover from the effects of 
transient faults. More than their analog predecessors, digital flight control systems are vul- 
nerable to external phenomena that can temporarily afTect the system without permanently 
damaging the physical hardware. External phenomena such as electromagnetic interference 
(EMI) can flip the bits in a processor’s memory or temporarily affect an ALU. EMI can 
come from many sources such as cosmic radiation, lightning or High Intensity Radiated 
Fields (IIIRF). There is growing concern over the effects of IIIRF on flight control systems. 
In the FA A Digital Systems Validation Handbook volume II [5], we find: 

A number of European military aircr aft fa tal accidents have been attributed to 
High Energy Radio Frequency (IIERF). 2 A digital fly-by- wire military Tornado 
aircraft and crew were lost during a tactical training strafing attack in Germany. 

The loss was attributed to IIERF when the aircraft flew through a high intensity 
Radio Frequency (RF) field. The civil/military aviation industry has very limited 
experience or data directed to accidents caused by electromagnetic transients 
and/or radiation. The present criteria, specifications, and procedures are being 
reevaluated. The IIERF fields apparently upset the digital flight control system 
of the Tornado which was qualified to a very low electromagnetic Environment 
(EME) standard. 

While composite materials may offer significant advantages in strength, weight, 
and cost, they provide less electromagnetic shielding than aluminum. The use 

'In fly-by-wire aircraft the direct mechanical and hydraulic linkages between the pilot and actuators of 
the system are replaced with digital computers. These digital computers are being used to control life critical 
functions such as the engines, sensors, fuel systems and actuators. 

2 The term ITERF has largely been replaced in current usage by the newer term IIIRF. 
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of solid-state digital technology in flight-critical systems create major challenges 
to prevent transient susceptibility and upset in both civil and military aircraft. 
Therefore, the Civil Aviation Authority (CAA), United Kingdom (U.K.) and 
the Federal Aviation Administration (FAA), United States (U.S.) voiced concern 
relative to emerging technology aircraft and systems. 

The RCP system is designed to automatically flush the effects of transients periodically, as 
long as the effect of a transient is not massive, that is, simultaneously affecting a majority 
of the redundant processors in the system. Of course, there is no hope of recovery if the 
system designed to overcome transient faults contains a design flaw. Consequently, a major 
emphasis in this work has been the development of techniques that mathematically show 
when the desired recovery properties have been achieved. The advantages of this approach 
are significant: 

• Confidence in the system does not rely primarily on end-to-end testing, which can 
never establish the absence of some rare design flaw (yet more frequent than 10 9 [6]) 
that can crash the system [7]. 

• Minimizes the need for experimental analysis of the effects of F,MI or HIRF on a digital 
processor. The probability of occurrence of a transient fault must be experimentally 
determined, but it is not necessary to obtain detailed information about how a transient 
fault propagates errors in a digital processor. 

• 'J'he role of experimentation is determined by the assumptions of the mathematical 
verification. The testing of the system can be concentrated at the regions where the 
design proofs interface with the physical implementation. 


1.1 Design of the Reliable Computing Platform 

Traditionally, the operating system function in flight control systems has been implemented 
as an executive (or main program) that invokes subroutines implementing the application 
tasks. For ultra- reliable systems, the additional responsibility of providing fault tolerance 
and undergoing validation makes this approach questionable. We propose a well-defined 
operating system that provides the applications software developei a ieliable mechanism for 
dispatching periodic tasks on a fault-tolerant computing base that appeal's to him as a single 
ultra-reliable processor. 

Our system design objective is to minimize the amount of experimental testing required 
and maximize our ability to reason mathematically about correctness. The following design 
decisions have been made toward that end: 

• the system is non- recon figurable 

• the system is frame-synchronous 

• the scheduling is static, non- preemptive 

• internal voting is used to recover the state of a processor affected by a transient fault 

3 Future work will concentrate on the massive transient and techniques to detect and restart a massively 
upset system. 


2 




Figure 1: Hierarchical Specification of the Reliable Computing Platform. 

A four-level hierarchical decomposition of the reliable computing platform is shown in fig- 
ure 1. 

The top level of the hierarchy describes the operating system as a function that sequen- 
tially invokes application tasks. This view of the operating system will be referred to as the 
uniprocessor model , which is formalized as a state transition system in section 3.2 and forms 
the basis of the specification for the IlCP. 

Fault tolerance is achieved by voting results computed by the replicated processors op- 
erating on the same inputs. Interactive consistency checks on sensor inputs and voting of 
actuator outputs require synchronization of the replicated processors. The second level in 
the hierarchy describes the operating system as a synchronous system where each replicated 
processor executes the same application tasks. The existence of a global time base, an in- 
teractive consistency mechanism and a reliable voting mechanism are assumed at this level. 
The formal details of the model, specified as a state transition system, are described in 

section 3.3. 

Although not anticipated during the Phase 1 effort, another layer of refinement was in- 
serted before the introduction of asynchrony. Level 3 of the hierarchy breaks a frame into 
four sequential phases. This allows a more explicit modeling of interprocessor communication 
and the time phasing of computation, communication, and voting. T he use of this interme- 
diate model avoids introducing these issues along with those of real time, thus preventing 
an overload of details in the proof process. 

At the fourth level, the assumptions of the synchronous model must be discharged. 
Rushby and von Henke [8] report on the formal verification of Lamport and Melliar-Smith’s 
[9] interactive-convergence clock synchronization algorithm. This algorithm can serve as a 
foundation for the implementation of the replicated system as a collection of asynchronously 
operating processors. Dedicated hardware implementations of the clock synchronization 
function are a long-term goal. 

Final realization of the reliable computing platform is the subject of the Phase 3 effort. 
The research activity will culminate in a detailed design and prototype implementation. 
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Figure 2: Generic hardware architecture. 

Figure 2 depicts the generic hardware architecture assumed for implementing the replicated 
system. Single-source sensor inputs are distributed by special purpose hardware executing 
a Byzantine agreement algorithm. Replicated actuator outputs are all delivered in parallel 
to the actuators, where force-sum voting occurs. In ter processor communication links allow 
replicated processors to exchange and vote on the results of task computations. As previously 
suggested, clock synchronization hardware may be added to the architecture as well. 

1.2 Overview of Results 

Before presenting the complete details, we provide an overview of the major formalizations 
and results for the reliable computing platform. In accordance with accepted terminology, 
we consider a fault to be a condition in which a piece of hardware is not operating within 
its specifications due to physical malfunction, and an error to be an incorrect computation 
result or system output. When a fault occurs, errors may or may not be produced. Although 
fault-tolerant architectures offer a high degree of immunity from hardware faults, there is a 
limit to how many simultaneous faults can be tolerated. Unless this limit is exceeded during 
system operation, the system will mask the occurrence of errors so that the system as a 
whole produces no computation errors. If the limit is exceeded, however, the system might 
produce erroneous results. 
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The primary mechanism for tolerating faults is voting of redundant computation results. 
Voting can take place at a number of locations in the system and associated with each choice 
are various tradeoffs. If voting occurs only at the actuators and the internal state of the 
system (contained in volatile memory) is never subjected to a vote, a single transient fault can 
permanently corrupt the state of a good processor. I his is an unacceptable approach since 
field data indicates that transient faults are significantly more likely than permanent faults 
[10]. An alternative voting strategy is to vote the entire system state at frequent intervals. 
This approach quickly purges the effects of transient faults from the system; however, the 
computational overhead for this approach may be prohibitive. There is a trade-off between 
the rate of recovery from transient faults and the frequency of voting. The more fiequent the 
voting, the faster the recovery from transients, but at the price of increased computational 
overhead. We observe that voting need only occur for a system state that is not recoverable 
from sensor inputs. A sparse voting approach can accomplish recovery from the effects of 
transient faults at greatly reduced overhead, but involves increased design complexity. The 
formal models presented here provide an abstract characterization of the voting requirements 
for a fault-tolerant system that purges the effects of transient faults. 

The proofs we construct are implicitly conditional to account for the situation of limited 
fault tolerance. The main results we establish can be expressed by the following formula. 

%...,A)D* = V(r ,r n ) 

where W is a predicate to define a minimal working hardware subset over time, s is the 
uniprocessor model’s system results, r, , . . . , r n are the results of the replicated processors, and 
V is a function that selects the properly voted values at each step. Moreover, asynchronous 
operation is assumed at the lowest specification layer. In this case, we further establish that 
if the minimal working hardware includes an adequate number of nonfaulty clocks, and clock 
synchronization is maintained, then the voted outputs continue to match those of higher level 
specifications. Thus, as long as the system hardware does not experience an unusually heavy 
burst of component faults, the proof establishes that no erroneous operation will occur at 
the system level. Individual replicates may produce errors, but they will be out-voted by 
replicates producing correct results. 

If the condition W were true 100% of the time, the system would never fail. Unfortu- 
nately, real devices are imperfect and this cannot be achieved in practice. The design of 
the fault-tolerant, architecture must ensure that condition W holds with high probability; 
typically, the goal is P(W) > 1 - 10 -9 for a 10 hour mission. This condition provides a 
vital connection between the reliability model and the formal correctness proofs. The proofs 
conditionally establish that system output is not erroneous as long as W holds, and the 
reliability model predicts that W will hold with adequately high probability. 

In the formal development to follow, we model the possible occurrence of component 
hardware faults and the unknown nature of computation results produced under such condi- 
tions. It is important to note that this modeling is for specification purposes onhj and reflects 
no self-cognizance on the part of the running system. We assume a nonreconfigurable archi- 
tecture that is capable of masking the effects of faults, but makes no attempt to detect or 
diagnose those faults. Each replicate is computing independently and continues to operate 
the best it can under faulty conditions; it has no knowledge of its own faultiness or that of 
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its peers. Wherever the formal specifications consider the two cases of whether a processor 
is faulty or not, it is important to remember that this case analysis is not performed by the 
running system. Also, it is important to realize that transient-fault recovery is a process 
that is continually in effect, even when there have been no fault occurrences. Each processor 
in the system continually votes and replaces its state with voted values. Thus, the transient 
fault recovery process does not require fault detection. 


1.3 Previous Efforts 


Many techniques for implementing fault-tolerance through redundancy have been developed 
over the past decade, e.g. SIFT [11], FTMP [12], FTP [13], MAFT [14], and MARS [15], 
An often overlooked but significant factor in the development process is the approach to 
system verification. In SIFT and MAFT, serious consideration was given to the need to 
mathematically reason about the system. In FTMP and FTP, the verification concept was 


almost exclusively testing. 

Among previous efforts, only the SIFT project attempted to use formal methods [16]. Al- 
though the SIFT operating system was never completely verified [17], the concept of Byzan- 
tine Generals algorithms was developed [18] as was the first fault-tolerant clock synchroniza- 
tion algorithm with a mathematical performance proof [9]. Other theoretical investigations 
have also addressed the problems of replicated systems [19]. 

Some recent work at SRI International has focused on problems related to the style of 
fault-tolerant computing adopted by RCP. Rushby has studied a fault masking and tran- 
sient recovery model and created a formalization of it using ElIDM [20, 21]. In addition, 
Shankar has undertaken the formalization of a general scheme for modeling fault-tolerant 
clock synchronization algorithms [22, 23]. 


2 Specification Hierarchy and Verification Approach 

This section outlines the general methods used in the RCP specifications and proofs. Detailed 
discussions of the actual specifications appear in later sections. 


2.1 The State Machine Approach to Specification 

The specification of the Reliable Computing Platform (RCP) is based upon a state-machine 
method. The behavior of the system is described by specifying an initial state and the 
allowable transitions from one state to another. The specification of the transition must 
determine (or constrain) the allowable destination states in terms of the current state and 
current inputs. One way of doing this is to specify the transition as a function. 

ftran '■ stale x input — > state 

This is an appealing method when it can be used. A second method is to specify the transition 
as a mathematical relation between the current state, the input and the new state. One way 
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to specify a mathematical relation is to define it using a function from the current state, the 
current input and the new state to a boolean: 

R : state x input x state — » boolean 

The function R is true precisely when the relation holds and false, otherwise. The meaning 
is as follows: a transition from the current state to the new state can occur only when the 
relation is true. Although the concept is simple it is somewhat awkward to use at first. 
Consider the function g defined by g(x) = (x + 4) 2 . 

In relational form this function might be expressed by: 

R{x,y) = [y = (a: + 4) 2 ] 

The latter form is more awkward than the former when a purely functional relationship exists 
between x and y. However, a relational approach has some advantages over a functional 
approach for the specification of complex system behavior. In particular, nondeterminism 
can be accommodated in a specification by only partially constraining system behavior. For 
example, if R is changed to the following: 

R(x,y) = \x > 0 D y = (x + A) 2 } 

the value of y is specified ordy for positive values of x. In other cases, any value of y would 
stand in the relation R to x. Such partially constrained specifications are very natural for 
modeling fault- tolerant systems. It allows us to say nothing about the behavior of failed 
components, thereby enabling proved results to hold no matter what behavior is exhibited 
by failed components during system operation. 

The relation R would be described as follows in the ElIDM specification language: 

R: funct ion [number , number -> bool] = 

(LAMBDA x,y : (x > 0 IMPLIES y = (x+4)*(x+4))) 

The first line declares that R, is a function from number x number to the set of booleans 
(bool). The second line uses lambda notation to define the body of the function. 

It should also be noted that the modeling approach used in this paper is not based upon 
a finite state machine technique. Some of the components of the state takes values from 
infinite domains. Therefore, verification tools such as STATEMATE [24] or MCB [25] are 
not applicable to our specifications. 

2.2 Specifying Behavior in the Presence Of Faults 

The specification of the RCP system is given in relational form. This enables one to leave 
unspecified the behavior of a faulty component. Consider the example below. 

R tTan : function[State, State — > bool] = 

(As, / : nonfaulty(s(r)) D l{i) = f(s{i))) 
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In the relation f? tran , if component i of state .s is nonfaulty, then component i of the next 
state t is constrained to be equal to f(s(i)). For other values of t, that is, when s(i) is faulty, 
the next state value t(i) is unspecified. Any behavior of the faulty component is acceptable 
in the specification defined by Rtran- 

An alternative approach is to define the transition as a partially-specified function: 

/iron : function[State — ► State] 

tran.ax : Axiom nonfaulty(.s(i)) 3 f tran (*** )(0 = ffO’f*)) 

This approach does not fit within the definitional structure of EflDM. Therefore, one must 
use an axiom to specify properties of a total, but partially defined function. This leads to a 
large number of axioms at the base of the proofs and significantly increases the possibility 
of inconsistency in the axiom set. 

2.3 The Specification Hierarchy 

The RCP specification consists of four separate models of the system: Uniprocessor Sys- 
tem (US), Replicated Synchronous (RS), Distributed Synchronous (DS), Distributed Asyn- 
chronous (DA). Each of these specifications is in some sense complete; however, they are at 
different levels of abstraction and describe the behavior of the system with different degrees 
of detail. The US model is the most abstract and defines the behavior of the system using a 
single uninterpreted definition. The RS level supplies more detail. The computation is repli- 
cated on multiple processors but the data exchange and voting is captured in one transition. 
The next level, the DS level, introduces even more detail. Explicit buffers for data exchange 
are modeled and the transition of the RS level is decomposed into 4 sub-transitions. The DA 
level introduces time, and different clock times on each of the separate processors. 2 * 4 

1. Uniprocessor System layer (US). As in the Phase 1 report [1], this constitutes the 
top-level specification of the functional system behavior defined in terms of an idealized, 
fault-free computation mechanism. This specification is the correctness criterion to be 
met by all lower level designs. The top level of the hierarchy describes the operating 
system as a function that performs an arbitrary, application-specific computation. 

2. Replicated Synchronous layer (RS). This layer corresponds to level 2 of the Phase 1 
report. Processors are replicated and the state machine makes global transitions as if 
all processors were perfectly synchronized. Iuterprocessor communication is hidden and 
not explicitly modeled at this layer. Suitable mappings are provided to enable proofs 
that the RS layer satisfies the US layer specification. Fault tolerance is achieved using 
exact-match voting on the results computed by the replicated processors operating on 
the same inputs. Exact match voting depends on two additional system activities: 
(1) single source input data must be sent to the redundant sites in a consistent man- 
ner to ensure that each redundant processor uses exactly the same inputs during its 

‘‘Due to the difficulties associated with reasoning about asynchronous systems, it was desirable to perform 

as much of the design and verification using a synchronous model as possible. Thus, only at level 4 is time 
explicitly introduced. 
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computations, and (2) the redundant processing sites must synchronize for the vote. 
Interactive consistency can he achieved on sensor inputs by use of Byzantine-resilient 
algorithms [18], which are probably best implemented in custom hardware. To ensure 
absence of single-point failures, electrically isolated processors cannot share a single 
clock. Thus, a fault-tolerant implementation of the uniprocessor model must ultimately 
be an asynchronous distributed system. However, the introduction of a fault-tolerant 
clock synchronization algorithm, at the DA layer of the hierarchy, enables the upper 
level designs to be performed as if the system were synchronous. 

3. Distributed Synchronous layer (DS). Next, the interprocessor communication 
mechanism is modeled and transitions for the RS layer machine are broken into a 
series of subtransitions. Activity on the separate processors is still assumed to occur 
synchronously. Interprocessor communication is accomplished using a simple mailbox 
scheme. Each processor has a mailbox with bins to store incoming messages from each 
of the other processors of the system. It also has an outgoing box that is used to 
broadcast data to all of the other processors in the system. The DS machine must be 
shown to implement the RS machine. 

4. Distributed Asynchronous layer (DA). Finally, the lowest layer relaxes the as- 
sumption of synchrony and allows each processor to run on its own independent clock. 
Clock time and real time are introduced into the modeling formalism. The DA machine 
must be shown to implement the DS machine provided an underlying clock synchro- 
nization mechanism is in place. 

The basic design strategy is to use a fault-tolerant clock synchronization algorithm as the 
foundation of the operating system. The synchronization algorithm provides a global time 
base for the system. Although the synchronization is not perfect it is possible to develop 
a reliable communications scheme where the clocks of the system are skewed relative to 
each other, albeit within a strict known upper bound. For all working clocks p and q, the 
synchronization algorithm provides the following key property: 

\ c p(T)-c q (T)\<6 

assuming that the number of faulty clocks, say m, does not exceed (nrep-l)/3, where nrep 
is the number of replicated processors. This property enables a simple communications 
protocol to be established whereby the receiver waits until maxb -f- 8 after a pie-determined 
broadcast time before reading a. message, where maxb is the maximum communication delay. 

Each processor in the system executes the same set of application tasks every cycle. A 
cycle consists of the minimum number of frames necessary to define a continuously repeating 
task schedule. Each frame is frame.time units of time long. A frame is further decomposed 
into 4 phases. These are the compute, broadcast, vote and sync phases. During the compute 
phase, all of the applications tasks scheduled for this frame are executed. The results of all 
tasks that are to be voted this frame are then loaded into the outgoing mailbox. During 
the next phase, the broadcast phase, the system merely waifs a sufficient amount of time to 
allow all of the messages to be delivered. As mentioned above, this delay must be greater 
than maxb + 8. During the vote phase, each processor retrieves all of the replicated data 
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from each processor and perforins a voting operation. Typically, this operation is a majority 
vote on each of the selected state elements. The processor then replaces its local memory 
with the voted values. It is crucial that the vote phase is triggered by an interrupt and 
all of the vote and state-update code be stored in ROM. This will enable the system to 
recover from a transient even when the program counter has been affected by a transient 
fault. Furthermore, the use of ROM is necessary to ensure that the code itself is not afFected 
by a transient . 5 During the final phase, the sync phase, the clock synchronization algorithm 
is executed. Although conceptually this can be performed in either software or hardware, 
we intend to use a hardware implementation. 

2.4 Extended State Machine Model 

Formalizing the behavior of the Distributed Asynchronous layei rec|uiies a means of incor- 
porating time. We accomplish this by formulating an extended state machine model that 
includes a notion of local clock time for each processor. It also recognizes several types 
of transitions or operations that can be invoked by each processor. The type of operation 
dictates which special constraints are imposed on state transitions for certain components. 

time-extended state machine model we use allows for autonomous local clocks on 
each processor to be modeled using snapshots of clock time coinciding with state transitions. 
Clock values represent the time at which the last transition occurred (time current state was 
entered). If a state was entered by processor p at time T and is occupied for a duration D, 
the next transition occurs for p at time T + D and this clock value is recorded for p in the 
next state . 6 A function c p (T) is assumed to map local clock values for processor p into real 
time. c p (T) is a specification-only function; it is not implemented by the system. 

Clocks may become skewed in real time. Consequently, the occurrence of corresponding 
events on different processors may be skewed in real time. A state transition for the DA 
state machine corresponds to an aggregate transition in which each processor experiences 
a particular event, such as completing one phase of a frame and beginning the next. Each 
processor may experience the event at different real times and even different clock times if 
duration values are not identical. 

The DA model is based on a specialized kind of state machine tailored to the needs of 
an asynchronous system ol replicated processors. The intended interpretation is that each 
component of the state models the local state of one processor and its associated hardware. 
Each processor is assumed to have a local clock running independently of all the others. 
Interprocessor communication is achieved by one class of transition that performs a simulta- 
neous broadcast of a portion of the local state variables to all the other processors. Broadcast 
values are assumed to arrive in the destination mailboxes within a bounded amount of real 
time maxb. 

The four classes of transitions are defined as follows: 

Rn the design specifications, these implementation details are not explicitly specified. However, it is clear 
that in order to successfully implement the models and prove that the implementation performs as specified, 
such implementation constructs will be needed. These issues will he explored in detail in future work. 

6 \Ve will use the now standard convention of representing clock time with capital letters and real time 
with lower case letters. 
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Figure 3: States, transitions, and mappings. 


1. L: Purely local processing that involves no broadcast communication or reading of the 
mailboxes. 

2. B: Broadcast communication where a send is initiated when the state is entered and 
must bo completed before the next transition. 

3. R: Local processing that involves no send operations, but does include reading of 
mailbox values. 

4. C: Clock synchronization operations that may cause the local clock to be adjusted and 
appear to be discontinuous. 

We make the simplifying assumption that the duration spent in each state, except those of 
type C, is nominally a fixed amount of clock time. Allowances need to be made, however, for 
small variations in the actual clock time used by real processors. Thus if v is the maximum 
rate of variation and D h D A are the intended and actual durations, then \D A - Di\ < vDi 
must hold. 


2.5 The Proof Method 

The proof method is a variation of the classical algebraic technique of showing that a 
homomorphism exists. Such a proof can be visualized as showing that a diagram “commutes” 
(figure 3). The system is described at two levels of abstraction, which will be referred^ to 
as the top and bottom levels for convenience. The top level consists of a current state s', a 
destination state, V and a transition that relates the two. The properties of the transition 
are given as a mathematical relation, Afto P (s',t')- Similarly, the bottom level consists of a 
state s , a destination state, l and a transition that relates the two. The properties of the 
transition are given as a mathematical relation, Afbottom{s, 0- The state values at the bottom 
level are related to the state values at the top level by way of a mapping function, map. To 
establish that the bottom level implements the top level one must show that the diagram 

commutes: 

D M u,p{ rnap(s), map(t)) 
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Figure 4: The RCP state machine and proof hierarchy 

where map(s) = s' and map(t) = t ' in the diagram. One must also show that initial states 
map up: 

^ bottom ( s ) D Itop(rnnp(s)) 

An additional consideration in constructing such proofs is that only states reachable from 
an initial state are relevant. Thus, it suffices to prove a conditional form of commutativity 
that assumes transitions always begin from reachable states. A weaker form of the theorem 
is then called for: 


reach able(s) A Af bo ttom{s , 0 3 J^to P (map(s),map(t)) 

This form enables proofs that proceed by first establishing state invariants. Each invariant 
is shown to hold for all reachable states and then invoked as a lemma in the main proof. 

Figure 4 shows the complete state machine hierarchy and the relationships of transitions 
within the aggregate model. By performing three layer-to-layer state machine implementa- 
tion proofs, the states of DA, the lowest layer, are shown to correctly map to those of US, 
the highest layer. This means that any implementation satisfying the DA specification will 
likewise satisfy US under our chosen interpretation. 
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3 US/RS Specification 

Up to now we have dealt only with general methods. Next we present the RCP specifications 
as developed using the EHDM language. An index at the end of this report indicates page 
numbers where each specification identifier and special symbol is defined in the text. The 
complete ElIDM specifications can be found in Appendix A. 

3.1 Preliminary Definitions 

The US and RS specifications are expressed in terms of some primitive type definitions. 
First, we must establish a “domain” or type to represent the complete computation state of 
a processor. This domain is called Pstate. It is declared in ElIDM as 

Pstate: Type (* computation state of a single processor *) 

Thus, all of the state information subject to computation has been collapsed into a single 
type Pstate. Similarly, inputs denotes the domain of external system inputs (sensors), and 
outputs the domain of output values that will be sent to the actuators of the system. These 
domains are named by the following EHDM declarations: 

inputs: Type (* type of external sensor input *) 
outputs: Type (* actuator output type *) 

The number of processors in the system is declared as an arbitrary, positive constant, nrep: 

nrep: nat (* number of replicated processors *) 

The constraint on n rep’s value is expressed by the following axiom 

processors.exist.ax: Axiom nrep > 0 

is a requirement that the system have at least one processor. Nearly all symbolic constants 
we introduce will have similar constraints imposed on them. 

At the RS level and below, information is exchanged among processors via some interpro- 
cessor communication mechanism. Additional types are needed to describe the information 
units involved, being based on a mailbox model of communication. First, we introduce a 
domain of values for each bin in the mailboxes: 

MB : Type (* mailbox exchange type *) 

Then we construct a type for a complete mailbox on a processor: 

MBvec: Type — array [processors] of MB 

This scheme provides one slot in the mailbox array for each replicated processor. 
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3.2 US Specification 

The US specification is very simple: 


Var Pstate 
u: Var inputs 

M ua : Definition function[Pstate, Pstate, inputs -> bool] = 

( \s,t,u:t = fc{u,s)) 


The function Af U a defines a mathematical relation between a current state and a final state, 
i.e., it defines the transition relation. For this model, the transition condition is captured by a 
function: / c (u,s), i.e., the computation performed by the uniprocessor system is determinis- 
tic and thus can be modeled by a function f c : inputs x Pstate -> Pstate. To fit the relational, 
nondeterministic state machine model we let the state transition relation A r us (s,<,u) hold iff 


t = f c (u,s). 

External system outputs are selected from the values computed by f c . The function 
• Pstate — > outputs denotes the selection of state variable values to be sent to the actuators. 
The type outputs represents a composite of actuator output types. 

Although there is no explicit mention of time in the US model, it is intended that a 
transition correspond to one frame of the execution cycle (i.e., the schedule). 

The uninterpreted constant initial_proc_state represents the initial Pstate value from which 
computation begins. 


initiaLus: function[Pstate — ► bool] = ( A s : s — initial_proc_state) 


initiaLus is expressed in predicate form for consistency with the overall relational method of 
specification, although in this case the initial state value is unique. 


3.3 RS Specification 

At the RS layer of design, the state is replicated and a postprocessing step is added after 
computation. This step represents the voting of state variables and thus may be selectively 
applied. It suffices to encapsulate the entire voting process under a single function of the 
global state. Nonetheless, it is better to split voting into two parts to facilitate refinement to 
the DS layer. Another difference introduced at this layer is that the state transition relation 
needs to be conditioned on the uonfaulty status of each processor. 

The global state at this level has type RSstate. This is a vector of length nrep where 
each component of the vector defines the state of a specific processor. Each processor in 
the system can be faulty or nonfaulty as a function of time measured in frames. The local 
processor “state” must not only reflect the computation state but indicate whether or not 
a processor is faulty. Such status information about faultiness is included for the purpose 
of modeling system behavior. An actual system component would be unable to maintain 
this status and it is understood that this part of the state exists only to model operational 
behavior and is not an implemented part of the system. Specification of the state type is as 
follows: 
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rs.proc-state: Type = Record healthy : nat, 

proc_state : Pstate 
end record 

RSstate: Type = array [processors] of rs_proc_state 

The state of a single processor is given by a record named rs_proc_state. The first field of 
the record is healthy, which is 0 when a processor is faulty. Otherwise, it indicates the 
(unbounded) number of state transitions since the last transient fault. Its value is one 
greater than the number of prior nonfaulty frames. A permanent fault is indicated by a 
perpetual value of 0. A processor that is recovering from a transient fault is indicated by a 
value of healthy less than the recovery period, denoted by the constant recovery .period. This 
constant is determined by details of the application task schedule and the voting pattern used 
for transient recovery. A processor is said to be working whenever healthy > recovery .period. 
The second field of the record is the computation state of the processor. It takes values from 
t he same domain as used in the US specification. The complete state at this level, RSstate, 
is a vector (or array) of these records. 

Two uninterpreted functions are assumed to express specifications that involve selective 
voting on portions of the computation state. Their role is described more fully in section 3.5. 

f a : function[Pstate -* MB] (* state selection for voting *) 

/„: fu nction [Pstate, MBvec -»■ Pstate] (* voting and overwriting *) 

These two functions split up the selective voting process to mirror what happens in the RCP 
architecture. First, /, is used to select a subset of the state components to be voted during 
the current frame. The choice of which components to vote is assumed to depend on the 
computation state. It maps into the type MB, which stands for a mailbox item. Second, 
the function /„ takes the current state value and overwrites selected portions of it with 
voted values derived from a vector of mailbox items. Voting is performed on a component- 
by-component basis, that is, applied to each task state separately, rather than applied to 
entire mailbox contents. Note that selection via /., need not be a mere projection, but could 
involve more complex data transformations such as adding checksums to ensure integrity 
during transmission. 

Given this background, the transition relation, M Tay can be defined: 

Af rs : Definition function[RSstate, RSstate, inputs — » bool] = 

( A s,f,« : ( 3 h : ( V* : 

(s(i)). healthy > 0 

D good.values.sent(s, u, h(i)) A voted_final_state(s, t , u, h, r))) 

A allow?ible_faults(s, t )) 

This relation is defined in terms of three subfunctions: good.values.sent, voted.finaLstate, 
and allowable-faults. The first aspect of this definition to note is that the relation holds 
only when allowable-faults is true. This corresponds to the “Maximum Fault Assumption” 
discussed in [I], namely that a majority of processors have been working up to the current 
time. The next thing to notice is that the transition relation is defined in terms of a. conjunc- 
tion good.values_sent(s,u,h(i)) A voted-final_state(s 1 t.u,h,i))). The meaning is intuitive: the 
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outputs produced by the good processors are contained in the vector h (i.e., fi(i) is derived 
from the value produced on processor z), and the final state t is obtained by voting the h 
values. Let us look at the voted .final-state relation first,. 

voted.finaLstate: function[RSstate, RSstate, inputs, MBmatrix, processors -> bool] 

= ( A s,t,u,h,i : <(*). proc-state = f v (f c (u, s{i). prostate), h(i))) 

Processor i is initially in state s(i). If it is nonfaulty (s(i). healthy > 0), then its transition 
to the state t(i) observes the following constraint: 

<(i). proc-state = f v (f c (u, s(i).procstate), h(i))) 

Otherwise, the behavior of the processor is not defined (i.e., a known mathematical relation 
is not given). The change to the processor state is defined using two functions, fofv The 
function f c is the same function used in the US specification. The function f v operates on 
the updated computation stale and values obtained from the other processors to produce a 
new state. The idea is that the new state is obtained by replacing local values with voted 
values. 

The values sent by the other processors must satisfy the following relation: 

good_values_sent: function[RSstate, inputs, MBvec — ► bool] = 

( A s,u,w : ( Vj : 

(«'(;)). healthy > 0 D w{j) = /,(/ e (u, s(j).proc_state)))) 

This relation constrains the h(i ) values used in the definition of the Af ra transition relation. 
Although this function is called with h(i) as an argument, its formal parameter is named w. 
There is one w value for each processor, which is used to model that processor’s mailboxes. 
If the sending processor j is nonfaulty (s(j). healthy > 0), then the value in the receiving 
mailbox w is given by 

/5(/c(«,^(;)-P' , oc- state ))- 

'Plie function f s selects which portion of the total state is to be voted. Note that since it is 
a function of the (complete) state, it can differ as a function of the frame, i.e., different data 
are voted during different frames. 

The allowable-faults function is defined as follows: 

allowable-faults: function[RSstate, RSstate — > bool] = 

(A s,t : maj_working(<) 

A (Vi : /(?'). healthy > 0 D l(i). healthy = 1 + s(i).healthy)) 

This function enforces the restriction imposed by the Maximum Fault Assumption, namely 
that all reachable states must have a majority of working processors. The condition is 
expressed in terms of the function maj-working and its subordinates: 

maj.condition: function[set[processors] — + bool] = 

( A A : 2 * card(A) > card(fullset[processors])) 
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working-proc: function[RSstate, processors — *• bool] — 

(A s,p: (s(p)). healthy > recovery.period) 

working-set: function [RSstate setfprocessors]] = 

( A s : ( A p : working.proc(.s,p))) 

maj-working: function [RSstate — + bool] = 

( A t : maj_condition(working_set(£))) 

The working-set function gives the set of working processors for the current replicated state. 
The cardinality of this set is then the number of working processors. (Note that sets are 
usually represented in EHDM by predicates on the element type. Thus, (Ax : P(x)) denotes 
the set {x\P(x)}.) The relation allowable-faults is defined whenever the destination state 
contains a majority of working processors. It also states that if a processor is nonfaulty for 
the current frame then the next state's value of healthy equals the previous state s value 
plus one. 

The initial state predicate initial.rs sets each element of the RS state array to the same 
value with the healthy field equal to recovery.period and the proc-state field equal to im- 
tiaLproC-State. 

initial.rs: function[RSstate — + bool] = 

(As : ( Vp : s(p). healthy = recovery.period As(p). proc-state = initial_proc_state)) 

The constant recovery-period is the number of frames required to fully recover a processor’s 
state after experiencing a transient fault. By initializing all healthy fields to this value, we 
are starting the system with all processors working. 

3.4 Actuator Outputs 

The nature of actuator outputs in the RCP application deserves special attention. In the 
uniprocessor case, an output is produced during each frame and sent to the actuators and 
no ambiguity exists. In a replicated system, however, multiple actuator values are produced 
and sent during each frame. Each nonfaulty processor p sends actuator values given by 
y‘ o (j-,s(p).proc_state). There are nrep sets of actuator values delivered in parallel, some of 
which may be copies of previous values for processors that have failed in such a way as to 
stop generating new values. 

It is understood that actuator outputs may be sent through one or more hardware voting 
planes before arriving at the actuators themselves. Other types of signal transformations 
may be applied to actuator lines between the output drivers and termination points. Ad- 
ditionally, some kind of force-sum voting typically is applied at the actuators to mask the 
presence of errors in one or more channels. All of this activity seeks to ensure that actuators 
perform as directed by a consensus of processors. These special-purpose requirements of the 
application leave us unable to completely reflect the proper constraints in the correctness 
criteria. However, we can use the majority function to map replicated output values into the 
single actuator output value that would be produced by an ideal uniprocessor. This captures 
the effect of voting planes and approximates the effect of force-sum voting at the actuators. 
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To show that replicated actuator outputs can be mapped into a single actuator output, we 
reason as follows. At the RS level, there are nrep actuator values given by / a (rs(p).proc-State) 
f or p _ . . . ? nrep. In section 4, a property of RS states is described that asserts that 

a majority exists among the proc_state values. In other words, a majority of values in 
{r.s(p).proc_state} equal maj(rs). Therefore, a majority of f a (rs{p). proc-state) values exists 
and is equal to / a (maj(rs)). Since maj(rs), the mapped value of an RS state, is equal to the 
corresponding US state, this shows that a majority of RS actuator outputs match the value 
produced by the fault-free US machine. 

Note that various additional requirements may be necessary, but are regarded as peculiar 
to the nature of an RCP application. Hence they must be imposed as correctness criteria 
beyond those necessary to show that one slate machine properly implements another. The 
intended use of replicated actuator outputs is not contained in the state machine models and 
may necessitate the use of additional, application-specific correctness conditions. 

3.5 Generic Fault- Tolerant Computing 

To model a very general class of fault- tolerant, real-time computing schemes, we seek to 
parameterize the specifications as much as possible. This parameterization takes the form 
of a set of uninterpreted constants, types, and functions along with axioms to constrain 
their values. Some instances have already been introduced. The function / c , for example, 
represents any computation that can be modeled as a function mapping from inputs and 
current state into a new state. As hardware redundancy and transient fault recovery are 
added to the specifications, additional types and functions are needed to express system 
behavior. 

3.5.1 State Model for Transient Fault Recovery 

Thus far, we have not concerned ourselves with the internal structure of the computation 
state Pstate. However, to capture the concept of recovering this state information piecewise, 
it is necessary to make some minimal assumptions about the structure of a Pstate value. 

control .state: Type (* portion of state used to control or schedule 

computation activities, e.g., frame counter *) 
cell: Type (* index for components of computation state *) 
cell-state: Type (* information content of computation state components *) 

We assume the state contains a control portion, used to schedule and manage computation, 
and a vector of cells , each individually accessible' and holding application-specific state in- 
formation. A sample instantiation of these types is that found in our previous report [1]: 
the control state is a frame counter and the cells represent the outputs of task instances in 
the task schedule. Unlike our previous model, however, the more general framework allows 
a system to maintain state information further back than just the previous execution of a 
schedule cell. 

Also assumed is the existence of access functions to extract and manipulate these items 
from a Pstate value. 
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succ: function [control_state -> controLstate] (* next control state *) 
f k ; function[Pstate -4 controLstate] (* extracts control state *) 

/ t : function[Pstate, cell — > celLstate] (* extracts cell (e.g. task) state *) 

As described in section 3.3, two additional functions are assumed to express specifica- 
tions that involve selective voting on portions of the computation state. The functions 
f a : Pstate — > MB and /„ : Pstate x MBvec — ♦ Pstate were introduced to model the selective 
voting process applied by each processor. /, selects which portions of the computation re- 
sults are subject to voting. f v takes these selected values from the replicated processors and 
replaces the required portions of the current state with voted values. 

For every voting scheme used for transient fault recovery within RCP, we must be able to 
determine when the state components have been recovered from voted values. This condition 
is expressed in terms of the current control state anti the number of nonfaulty frames since 
the last transient fault. Two uninterpreted functions arc provided for this purpose. 

rec: function[cell, control_state, nat — ► bool] 

The predicate rcc.{c, I<, II) is true ilT cell c’s state should have been recovered when in control 
state I< with healthy frame count II. Recall that we use a healthy count of one to indicate 
that the current frame is nonfaulty, but the previous frame was faulty. This means that 
IJ — | healthy frames have occurred prior to the current one. 

dep: function[cell, cell, control-state -4 bool] 

The predicate dcp(c,d, K) indicates that cell c’s value in the next state depends on cell d' s 
value in the current state, when in control state l\ . This notion ol dependency is different 
from the notion of computational dependency; it determines which cells need to be recovered 
in the current frame on the recovering processor for cell c’s value to be considered recovered 
at the end of the current frame. If cell c is voted during I\, or its computation takes only 
sensor inputs, there is no dependency. If c is not computed during K, c depends only on its 
own previous value. Otherwise, c depends on one or more cells for its new value. 

One derived function is used in the axioms. It asserts that two states X and Y agree on 
all the corresponding cells on which cell c depends. 

dep_agree: function[cell, control-state, Pstate, Pstate * bool] 

( A c, K, X, Y : ( V d : dep(c, d, K) D /,( X, d) = /,(K, d))) 

3.5.2 Transient Recovery Axioms 

Having postulated several functions that characterize a generic fault-tolerant computing 
application, it is necessary to introduce axioms that sufficiently constrain these functions. 
Once concrete definitions for the functions have been chosen, these axioms must be proved 
to follow as theorems for the RCP results to hold for a given application. The eight axioms 

are presented below. 

succ-ax: Axiom /*(/<-(«, ps)) = succ(/*(ps)) 
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The first axiom states the simple condition that f c computes the successor of its control state 
component. 

Three axioms give properties of the function rec. 

full-recovery: Axiom II > recovery-period D rec(c, K, II) 

initial-recovery: Axiom rec(c, K, II) D II > 2 

dep.recovery: Axiom rec(c, succ(A'), II + 1 ) Adep(c, d t K) D rec(d,K,II) 

First, we require that after the recovery period has transpired, all cells should be considered 
recovered by rec. Second, it takes a minimum of two frames to recover a cell. (This is 
necessary because one frame is used to recover the control state. In some applications, it 
may be possible to recover cells in one frame, but our proof approach does not accommodate 
those cases and the more conservative minimum of two is used.) Third, if cell c is to be 
recovered in the next state, all cells it depends on must be recovered in the current state. 

components_equal: Axiom 

MX) = MY) A (Vo : f t (X,c) = U(Y,c)) D X = Y 

This axiom, which is a type of extensionality axiom, requires that the control state and cell 
state values form an exhaustive partition of a Pstate value. 

Two axioms capture the key conditions for recovery of individual state components. 

control-recovered: Axiom 

maj-condition(yl) A ( Vp : p G A D iv(p) = /,( ps)) D fk{fv(Y, w)) = fk{ ps) 

cell-recovered: Axiom 
maj_condition(A) 

A ( V p : p € A D w(p) = f a (f c {u , ps))) 

A f k {X) = K A /fc(ps) = K A dep_agree(c, K, X, ps) 

3 ft(fv(fc(u,X),w),c) - f t (f c (u,ps),c) 

The first axiom requires that the control state component be recovered after every frame. 
Thus, /„ must vote the control state unconditionally and update the Pstate value accordingly. 
The conditions in the antecedent state that for a majority of processors, their mailbox items 
must match the value selected by the function /„. The other axiom gives the required 
condition for recovering an individual cell state value. All cell values that c depends on must 
already agree with the majority value. After voting with /„, the function f t must extract a 
cell state that matches that of the consensus. 

vote.maj: Axiom maj-Condition(A) A (Vp:p€ Ad w(p) = /,( ps)) 

D fv( ps,u>) = ps 

The final axiom expresses the additional requirement on f v that if a majority of processors 
agree on selected mailbox values derived from state p.s, then f v applied to ps preserves the 
value ps. In other words, once a Pstate value has been fully recovered, it will stay that way 
in the face of subsequent voting. 
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3.5.3 Sample Interpretations of Theory 

The proofs of seel, ion 4 make use of the foregoing axioms to establish that the RS specifica- 
tion correctly implements the US specification. A valid interpretation of the model provides 
definitions for the uninterpreted types and functions that are ultimately used to prove the 
axioms as theorems of the interpreted theory. To maintain the generality of our model and 
its applicability to a wide range of designs, we do not provide any standard interpreta- 
tions. Nevertheless, it is desirable to carry out the exercise to establish that the axioms are 

consistent and can be satisfied for reasonable interpretations. 

Two sample interpretations were constructed based on voting schemes introduced in 
the Phase 1 report [1]. Definitions for the basic concepts of a static, task-based scheduling 
system were formalized first. Included were the notions of cells as being derived from a frame, 
subframe pair, and state components to record both the frame counter as well as task outputs. 
Task execution according to a fixed, repeating schedule was assumed. Definitions were also 
provided for the continuous voting and cyclic voting schemes [I]. In both cases, the transient 
recovery axioms were proved using Ehdm. A preliminary form of these specifications are 

given in Appendix B. , . 

Carrying out the proofs required several changes to the module structure embodied in 

the specifications of Appendix A. For this reason, the specifications in Appendix B have not 
yet been integrated with the specifications of Appendix A. Additional work is required to 
integrate these provisional interpretations into the existing framework. The proofs conducted 
thus far were performed simply to demonstrate that the axioms could be satisfied and are 

thus consistent. , 

The continuous voting scheme requires that all state components are voted during each 

frame. Hence transient recovery is nearly immediate. Formalizations for this case are very 
simple and the proofs are trivial. The cyclic voting scheme represents the typical case where 
state components are voted in the frame they are produced. A cell’s value is not voted during 
frames where it is not recomputed. Formalization in this case is somewhat more involved 
and the proofs require a bit more effort. The proofs and supporting lemmas comprise about 
two pages of EHDM specifications. A few selected definitions for the cyclic voting functions 

are shown below. 

/,: function[Pstate — > MB] = 

(A ps : ps with [(control) := ps. control, (cells) := 
cell_apply(( A c : ps.cells(e)), 
ps. control, 
nulLcelLarray, 
num.cells)]) 

/„: function [Pstate, MBvec — + Pstate] = 

( A ps, w : ps with [(control) := k_maj(te), (cells) := 

cell_apply(( A c : t.maj(u;,r)), 
ps. control, 
ps. cells, 
num_cells)]) 
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rec: function[cell, control-state, nat — ► bool] = 

( A c, A', H : H 

> 1 + ( if K = cell-frame(e) 
then schedule-length 
else mod_minus( A', celhframe(c)) 
end if)) 

dep: function[cell, cell, control-state bool] = 

( A c, d, K : celLframe(c) £ K f\c — d) 

A few supporting definitions are omitted; these functions are presented merely to show the 
general order of complexity involved. 


4 RS to US Proof 

Proving that the RS state machine correctly implements the US state machine involves 
introducing a mapping between states of the two machines. The function RSmap defines the 
required mapping, namely the majority of Pstate values over all the processors. 

RSmap: function[RSstate -> Pstate] = ( A r.s : maj(rs)) 

maj: function[RSstate — ♦ Pstate] 
maj.ax: Axiom ( 3 A : 

maj.condition(A) A (Vp : p € A D (rs(p)). proc-State = us)) 

D maj(rs) = us 

The two theorems required to establish that RS implements US are the following. 

frame-commutes: Theorem reachable(s)AA/ r r ,(.s, t, u) D A r UJ (RSmap(s), RSmap(<), u ) 

initial-maps: Theorem initiaLrs(s) D initial_us(RSmap(s)) 

The theorem frame_commutes, depicted in figure 5, shows that a successive pair of reachable 
RS states can be mapped by RSmap into a successive pair of US states. The theorem 
initial-maps shows that an initial RS state can be mapped into an initial US state. 

The notion of state reachability is used to express the theorem fra me_com mutes. This 
concept is formalized as follows: 7 

rs.measure: function[RSstate, nat — > nat] == (\rs,k : k ) 
reachableJn_n: function [RSstate, nat — ► bool] = 

(\t,k : if fc = 0 

then initiaLrs(Z) 

else ( 3 s, u : reachable_in_n(s, k — 1 ) A Af ra ($, t, «)) 

end if) by rs_measure 

reachable: function[RSstate — * bool] = ( A i : ( 3 k : reachable_in_n(Z, &))) 
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Figure 5: Mappings in the RS to US proof. 

Proofs for the two main theorems are supported by a handful of lemmas. The most 
important is a state invariant that relates values of various state components to their corre- 
sponding consensus values. 

stateJnvariant: function[RSstate_prop — > bool] = 

(A rs.prop : (V< : reachable(f) 3 rs.prop(<))) 

state_rec_inv: Lemma state_invariant(state_recovery) 

control-recovery: function[RSstate — * bool] = 

(As: (Vp : (s(p)). healthy > 1 3 / 4 ((s(p)).proc_state) = /*(maj(s)))) 

cell-recovery: function [RSstate — > bool] = 

( A s : ( V p, c : 

rec (c, /*((s(p)).proc-State), (s(p)). healthy) 

3 /*((«(?))• proc_state, c) = /,(maj(s), c))) 

state-recovery: function[RSstate — * bool] = 

(As: maj.exists(s) A controLrecovery(s) A celLrecovery(s)) 

I’he invariant state_recovery is shown to hold for all reachable states. The control recovery 
condition of this invariant asserts that if a processor p has been nonfaulty for at least one 
frame, then the control state, as extracted by fk, is equal to the consensus value. Similarly, 
the cell recovery condition asserts that if cell c is due to be recovered, as indicated by the 
predicate rec, then cell state c, as extracted by ft, is equal to the consensus value. Proving 
the invariant requires invoking the axioms presented in section 3.5. 

Lemmas showing that a majority among RS state values continues to exist after every 
state transition are also proved in support of the invariant. One such lemma is also central 
to the proof of fra me.com mutes. 

TNote that functions defined with such as in rs.measure, are semantically equivalent to those 

(lefined with the only difference is automatic expansion of “==” functions during theorem proving. 
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rec_maj_f_c: Lemma 

maj.working(s) A state.recovery(s) A J\f r .,(s,t,u) D maj(f) = / c (w, maj(s)) 

With a majority of working processors and state.recovery holding in current state s, this 
lemma concludes that maj applied to the next state t equals the computation step f c applied 
to maj of s. From this lemma it is clear how RS slates and their images under maj will 
correspond to the desired US states. 

With the state recovery invariant established, most of the work needed to prove the main 
theorem frame.commutes is in hand. One additional lemma is useful to bridge the gap 
between the two. 

working-majority: function[RSstate — > bool] = 

( A .s : (Vp : p <E working.set(s) D (s(p)).proc_state = maj(s))) 

consensus-prop: Lemma state_recovery(.s) D working_majority(s) 

The lemma consensus-prop allows us to draw a key inference from the state.recovery invariant, 
which is expressed by the predicate working-majority. This predicate asserts that for all 
processors p that belong to the working set , i.e., for all working processors, p’s value of 
Pstate is equal to the majority value. 

The proof of fra me.com mutes now follows from rec.maj.f_c and consensus_prop and as- 
sorted definitions. The proof of initial-maps follows from definitions and the lemma ini- 
tial.maj.cond, which states that an initial state satisfies the majority condition. 

initial.maj.cond: Lemma initiaLrs(s) 3 maj_condition(working_set(s)) 

This completes the proof that the RS machine implements the US machine. 

Note that our proof is in terms of a generic model of fault-tolerant computation and 
depends on the validity of the axioms of section 3.5. For some choices of definitions for the 
uninterpreted functions, there will be substantial work required to establish those axioms 
as theorems. For example, the Minimal Voting scheme presented in our Phase 1 report [1] 
requires a nontrivial proof to establish that full recovery is achieved. Such details have been 
omitted here. Nevertheless, the value of our revised approach is in its generality. The results 
can now be made to apply to a wide variety of frame-based, fault-tolerant architectures. 


5 DS Specification 

In the Distributed Synchronous layer we focus on two things: expanding the state to include 
“mailboxes” for interprocessor communication and dividing a frame transition into four 
sequential subtransitions. The state must also be expanded to include an indicator of which 
phase of a frame is currently being processed. This is done as follows. 

The structure of the mailbox for a four- processor system is shown in figure 6. Each 
processor contains a mailbox with one slot dedicated to each other processor in the system. 
Each slot is large enough to contain the largest amount of data to be broadcast during one 
frame. The nth slot of processor « serves as the outgoing mailbox. 

The local stale for each processor can now be defined: 
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P4 


Figure 6: Structure of Mailboxes in a four-processor system 

ds_proc_state: Type = Record healthy : nat, 

proc_state : Pstate, 
mailbox : MBvec 
end record 

The vector of all processors ds_proc_state is named ds_proc_array: 

ds_proc_array: Type = array [processors] of ds_proc_state 

The complete DSstate is: 

DSstate: Type = Record phase : phases, 

proc : ds_proc_array 

end record 

In the DS specification, a frame is decomposed into four phases: 

phases: Type = (compute, broadcast, vote, sync) 

The first field of DSstate holds the current phase. During each phase a distinct function is 
performed. 

1. Computation. The proc_state component of the state is updated with the results of 
computation using the function f c . 

2. Broadcast. Interprocessor communication is effected by broadcasting the MB values 
to all other processors, which are deposited in their respective mailboxes. 
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3. Voting. The received mailbox values are voted and merged with the current Pstate 
values to arrive at the end-of-frame slate. 

4. Synchronization. The clock synchronization function is performed. (No details of 
the clocks are introduced until the DA specification layer.) 

The transition relation for the frame is defined in terms of a phase-transition relation 
Afds- 

frame.N.ds: function[D$state, DSstate, inputs — ► bool] = 

(A s,t,u :(3x,y,z: 

( s , X, u) A (x, y,u) A {y,Z,u)A u)) ) 

Note how the intermediate states are defined using existential quantifiers and that the output 
state of a phase transition becomes the input of the next phase transition. The net result of 
performing these four phase transitions will be shown to accomplish the same thing as the 
single transition of the RS specification. 

The phase-transition relation is defined as follows: 

Af d ,: function[DSstate, DSstate, inputs -+ bool] = 

( A s, /, u : maj-working(f) 

A i. phase = next_phase(s. phase) 

A ( V i : 

if s. phase = sync 
then AT da (s,t,i) 

else Aproc(i). healthy = s.proc(t). healthy 
A (s. phase = compute D Af da (s,t,u,i)) 

A (s. phase = broadcast D Afd s (s,t,i)) 

A (s. phase = vote D 0 ) 

end if)) 

Notice that the phase-transition relation only holds when the next state t has a majority 
of working processors. This corresponds to the analogous condition in M T3 presented in 
section 3.3, where it appears as one conjunct of the allowable-faults relation. Hence, all 
reachable states in the DS specification must have a majority of working processors. 

The phase field of the state is advanced by the function next_phase. The phase-transition 
relation is defined in terms of four sub-relations: A/£, N da , Af da , and A/^,, which correspond to 
the compute, broadcast, vote and sync phases, respectively. The quantifier Vt invokes the sub- 
relations for all of the processors of the system. Note that the statement f.proc(i). healthy = 
s.proc(i). healthy after the else requires that the value of healthy remain constant throughout 
a frame. Thus, if a processor is faulty anywhere in a frame it is considered to be faulty 
throughout; the value of healthy may only change at the frame boundaries, i.e., at the sync 
to compute transitions. Similarly, full recovery of state information does not occur until the 
end of a frame. This is consistent with the previous work [1]. 

Table 1 provides a summary of the functions that are performed during each phase on 
nonfaulty processors. In the table s, is an abbreviation for .s.proc(z). 

The A f d , sub-relation defines the behavior of a single processor during the compute phase: 


26 


Phase 

Held constant 

Modified 

compute 

healthy 

/.j. proc_state = /,.(«,. s;.proc_stale) 
Z;.mailbox(/‘) = /,(/ c (u,s,.proc_state)) 

broadcast 

proc-statc 

healthy 

(Vp : Z;.mailbox(p) = s p .mailbox(p)) 

vote 

mailbox 

healthy 

Z,. proc-state = proc-state, s,. mailbox) 

sync 

proo_state 

ti. healthy = 1 + .s,. heal thy 


Table 1: Summary of activities during various phases 

Mj a : function[DSstate, DSstate, inputs, processors — > bool] = 

( A s, t,u, i : 

s.proc(i). healthy > 0 

D Z. proc(r). proc-state = / c («, s.proc(;s). proc-state) 

A Z.proc(/).mailbox(i) = /,(/ c (u, .<s.proc(r). proc-state))) 

During this phase, the proc-State field is updated with the results of the computation: 

f c (i t, s.proc(i). proc-state) 

Also, the mailbox is loaded with the subset of the results to be broadcast as defined by the 
function /,. Recall that a processor’s own mailbox slot acts as the place to post outgoing 
data for broadcast to other processors. 

The Afj, sub- relation defines the behavior of a single processor during the broadcast phase: 

A/J,: function[D$state, DSstate, processors -» bool] = 

(A s,t,i : s.proc(i). healthy > 0 

D Z.proc(i). proc-state = s.proc(i). proc_state 
A broadcast_received(s, Z, *')) 

During this phase the proc_state field remains unchanged and the broadcast-received relation 
holds: 

broadcast-received: fu nction [DSstate, DSstate, processors — + bool] = 

(A.s,«,< 7 : (Vp : 

s. proc(p). healthy > 0 

D f.proc(<j').mailbox(p) = s.proc(p).mailbox(p))) 

'J’liis states that each non faulty processor q receives the values sent by other nonfaulty pro- 
cessors. If the sending processor p is faidty, then the consequent of the relation need not hold 
and the value found in p’s slot of q's mailbox is indeterminate. If the receiving processor q 
is faulty, the broadcast-received relation is not required to hold in A/*j,. In this situation, all 
of r/’s mailbox values are unspecified. 

The A sub-relation defines the behavior of a single processor during the vote phase: 
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Af£: function[DSstate, DSstate, processors — > bool] = 

(A s,L,i : s. proc(z). healthy > 0 

D Z.proc(i). mailbox = s.proc(i). mailbox 
A Z.proc(i).proc-State 

= /„(.s.proc(i).proc_state, s.proc(t). mailbox)) 

During this phase the mailbox field remains unchanged and the local piocessor state is up- 
dated with the result of voting the values broadcast by the other processors. The vote 
function is named 

The sub-relation defines the behavior of a single processor during the sync phase: 

function[DSstate, DSstate, processors — > bool] = 

(As,/, * : (s.proc(t). healthy > 0 

D Z.proc(i).proc-state = .s.proc(f).proc-State) 

A (/.proc(i). healthy > 0 

3 Z.proc(i). healthy = 1 + .s.proc(i). healthy)) 

During the sync phase, the computation state of a nonfaulty processor remains unchanged. 
At the end of the sync phase, the current frame ends, so the value of healthy is incremented 
by one if the processor is to be nonfaulty in the next frame. This is the same condition 
appearing in the relation allowable-faults of section 3.3. Any processor assumed to be faulty 
in the next frame will have its healthy field set to zero. A limit on how many processors 
can be faulty simultaneously is imposed by the predicate maj_working. Therefore, not every 
possible assignment of values to the healthy fields is admissible; each assignment must satisfy 
the Maximum Fault Assumption. 

The predicate initiaLds puts forth the conditions for a valid initial state. The initial 
phase is set to compute and each element of the DS state array has its healthy field equal to 
recovery-period and its proc_state field equal to initiaLproc_state. 

initiaLds: function[DSstate — * bool] = 

( A s : s. phase = compute 

A ( Vi : s.proc(i). healthy = recovery .period 

A s.proc(i).proc-state = initial_proc_state)) 

As before, the constant recovery.period is the number of frames required to fully recover a 
processor’s state after experiencing a transient fault. Ry initializing the healthy fields to this 
value, we are starting the system with all processors working. Note that the mailbox fields 
are not initialized; any mailbox values can appear in a valid initial DSstate. 


6 DS to RS Proof 

The DS specification performs the functionality of the RS specification in four sequential 
steps. Thus, we must show that the “frame” transition function, frame.N_ds, 

frame_N_ds(s, /, u) = ( 3 x,y,z : ;r,«)AA/j J (x,y,tt)AA/’,/ s (y, 2 ,u)AA^,(2:,/,u)) 

accomplishes the same function as a single transition of the RS level transition function 
Af rj (s,Z,tt) under an appropriate mapping function. 
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6.1 DS to RS Mapping 

The DS to RS mapping function, DSmap, is defined as: 

DSmap: function[DSstate -» RSstate] = ( A ds : ss_update(d.s, nrep)) 
where ss_update is given by: 

ss.update: Recursive function[DSstate, nat — + RSstate] = 

( A ds,p : if (p = 0)V(p> nrep) 
then rsO 

else ss_update(t/s,p — 1) 
with [(p) := rsprocO 

with [(healthy) := ds.proc(p). healthy, 

(proc_state) := d.s.proc(p).proc-State]] 
end if) by ssu-measure 

This mapping copies the healthy and proc_state fields for each processor as illustrated in 
figure 7. To establish that DS implements RS, the commutativity diagram of figure 8 must 



Figure 7: Mapping DS to RS: the DSmap function 

be shown to commute. To establish that the diagram commutes, the following formula must 
be proved. 

frame_commutes: Theorem 

s. phase = compute A frame_N_ds(.s, l , u) D A/’ r 4 (DSmap(s), DSmap(t), u ) 

Note that to make the correct correspondence, we must consider only DS states found at the 
beginning of each frame, namely those whose phase is compute. Refer to figure 4 on page 12 
for a visual interpretation of this theorem. 

It is also necessary to show that the initial states are mapped properly: 
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Figure 8: Commutative Diagram for DS to RS Proof 

initiaLmaps: Theorem initiaLds(s) D initiaLrs(DSmap(.s)) 

Several basic lemmas follow from the definition of the mapping function: 

map-1: Lemma DSmap(s)(i). healthy = .s.proc(i). healthy 
map_2: Lemma DSmap(s)(i).proc_state = s.proc(i).proc.state 
map_3: Lemma allowable_faults(s,i) D RS.allowable-faults(DSmap(s), DSmap(^)) 
map.4: Lemma RS.good_values_sent(DSmap(s),«, «;) = good_values_sent(s,u,iy) 

map-5: Lemma RS.voted_final_state(DSmap(s), DSmap(t),u, h, i) 

= voted_final_state(s, f , u, h, i) 

map.7: Lemma RS.maj_working(DSmap(s)) = DS.maj-working(s) 

6.2 The Proof 

The proof of the frame_commutes theorem involves the expansion of the frame.N.ds relation 
and showing that the resulting formula logically implies A/' r .,(DSmap(s), DSmap(f), u). We 
begin with the definition of frame_l\Lds: 

frame_N_ds(s,C«) = (3;r,2/,z : x, u) A Afda{x, y, u) A Afd s (y,z,u) A 

Since s. phase = compute, Af da (s,x,u) can be rewritten as: 

Af d3 (s,x,u ) = maj-working(x) A x. phase = broadcast 

A (Vi : x. proc(i). healthy = s.proc(i). healthy A J\f£ s (s y x,u,i)) 

Substituting for X ds (s,x,u) we obtain 
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s . phase — compute A frame_N_ds(.s, t, u) 

3 ( 3 x, y, z : maj_working(x) 

A ( Vj : x. phase = broadcast 

A x.proc(t). healthy = .s. proc(i). healthy A x, u, i)) 

A j y, «) A Mu(y, z , «) A ^ «)) 

Next, expand AJ,, the Md» term for the broadcast phase, and combine universal quantifiers: 

s . phase = compute A frame_N_ds(s, t, u) 

3 ( 3 x, y, z : maj-working(x) A maj.working(y) 

A (V* : x . phase = broadcast 

A x.proc(i). healthy = s.proc(z). healthy 
A (s.proc(i). healthy > 0 

3 x.proc(«).proc-state = / c (u,s.proc(z).proc_state)) 

A y. phase = vote 

A y.proc(t). healthy = x.proc(i). healthy 
A (x.proc(z). healthy > 0 

3 (?/.proc(j).proc_state = x.proc(z).proc-state 
A (V j : x.proc(j). healthy > 0 

3 j/.proc(z).mailbox(j) = / J (x.proc(j).proc_state))))) 

A Ndsiv, «) A Md*{z,t,u)) 

Simplifying to eliminate x yields: 

s . phase = compute A frame_N_ds(s, t, u) 

3 (3 y,z : maj_working(y) 

A ( V i : y. phase = vote 

A y.proc(t). healthy = s.proc(t). healthy 
A (s.proc(i). healthy > 0 

3 (y.proc(f).proc_state = / c (u, s.proc(z).proc_state) 

A (Vj : .s.proc(j). healthy > 0 

3 y.proc(z).mailbox(j) = /,((y.proc(;')).proc_state))))) 
A Md,{y , «) A Afd»(z,t,u)) 

Expanding the Md» term for the third phase and simplifying produces: 

s . phase = compute A frame_N_ds(.s, t , u ) 

3 ( 3 z : maj.working(z) 

A ( Vz : z . phase = sync 

A z.proc(i). healthy = .s.proc(i). healthy 
A (s.proc(i). healthy > 0 

3 z.proc(z).proc_state = / v (/ c («,s.proc(i).proc^tate), z.proc(t). mailbox) 
A ( V j : s.proc(ji). healthy > 0 

3 z.proc(i).mailbox(j) = /,(/ c (u, (s.proc(j)).proc_state))))) 

A Af<u{z,t,u)) 

Expanding the fourth phase Mda term and simplifying gives: 


'it**. 
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S. phase = compute A frame_N_ds(s, t, u) 

3 ( 3 2 : maj_working(f) 

A (Vi : f. phase = compute 
A (s.proc(i). healthy > 0 

3 f.proc(i).proc-state = /„(/ c (u, s.proc(i).proc_state), z.proc(i). mailbox) 
A ( V j : s.proc(j). healthy > 0 

3 z.proc(i).mailbox(j) = /*(/ c («, (s.proc(j)).proc_state)))) 

A (Lproc(i). healthy > 0 

3 Lproc(i). healthy = 1 + s.proc(i). healthy))) 

Letting h(i) = z.proc(i). mailbox, 

s . phase = compute A frame_N_ds(.s, t , u) 

3 maj_working(J) 

A ( 3 h : ( Vi : Lphase = compute 
A (f.proc(i). healthy > 0 

3 t.proc(i). healthy = 1 + s.proc(i). healthy) 

A (s.proc(i). healthy > 0 

3 Lproc(i).proc-State = /„(/ c (u, s.proc(i).proc_state), h(i)) 

A (Vj : s.proc(j). healthy > 0 

3 h(i)(j) = / 4 (/c(«,(-«-proc(j)).proc_state)))))) 

This must be shown to logically imply .A/" ra (DSmap(.s), DSmap(t),u), which can be rewrit- 
ten as: 

(3 h : ( V* : s.proc(i). healthy > 0 

3 (Vj : 3. proc(j). healthy >03 /»(i)(j) = /,(/ c («,^.proc(j).proc^tate))) 

A Lproc(i).proc_state = / u (/ c (u,3.proc(i).proc_state), /i(i)))) 

A allowable_faults(s, t)) 

The first conjunct can be seen to follow by inspection. By expanding allowable-faults, 

allowable-faults: function[RSstate, RSstate — * bool] = 

(A s t t: maj-working(t) 

A (Vi : t{i). healthy >03 (<(i)). healthy = 1 + s(i). healthy)) 
the second conjunct can be seen to follow as well. Q.E.D. 


7 DA Specification 

The DA specification performs the same functions as the DS specification; however, explicit 
consideration is given to the timing of the system. Every processor of the system has its own 
clock and consequently task executions on one processor take place at different times than 
on other processors. Nevertheless, the model at this level explicitly takes advantage of the 
fact that the clocks of the system are synchronized to within a bounded skew 6 . Therefore, 
it is necessary to give an overview of clock synchronization theory before elaborating the DA 
specification. 
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7.1 Clock Synchronization Theory 

In this section we will discuss the synchronization theory upon which the DA specification 
depends. Although the RCP architecture does not depend upon any particular clock synchro- 
nization algorithm, we have used the specification for the interactive consistency algorithm 
(ICA) [9, 8] since EHDM specifications for ICA already exist. 

In this section we show the essential aspects of this theory. The formal definition of a 
clock is fundamental. A clock can be modeled as a function from real time t to clock time T. 
c(t) — T or as a function from clock time to real time: c(7’) = t. Since the ICA theory was 
expressed in terms of the latter, we will also be modeling clocks as functions from clock time 
to real time. We must be careful to distinguish between an uncorrected clock and a clock 
which is being resynchronized periodically. We will use the notation c(T) for a uncorrected 
clock and rt^(T) to represent a synchronized clock during its ith frame. 8 

Good clocks have different drift rates with respect to perfect time. Nevertheless, this 
drift rate is bounded. Thus, we can define a good clock as one whose drift rate is strictly 
bounded by p/2. A clock is “good”, (i.e. a predicate good-dock (T 0 ,T n ) is true), between 
clock times To and T„ iff: 

( VT,, T a : To < T, < T„ A To < T 2 < T n 

D MT.) - C P (T 2 ) - (7\-M < f * [Ti - Tal) 

The synchronization algorithm is executed once every frame of duration frame-time. The 
notation T (,) is used to represent the start of the ith frame, i.e., (T° + i * frame-time). The 
notation T € means that T falls in the ith frame, i.e., 

( 3 IT : 0 < II < frame-time A T — + 11)) 

During the ith frame the synchronized clock on processor p, rt p , is defined by: 
rt p (i,T) = c p (T + Corrj?) 

where Corr is the cumulative sum of the corrections that have been made to the (logical) 
clock. It is defined by : 

Corr*’* = if i > 0 then Corr£ _1) + 
else initiaLCorr(p) 
end if 

where initiaLCorr(p) is conveniently equated to zero (i.e. Corr^ = 0). The function A*/ ^ 
is the correction factor for the current frame as computed by the clock synchronization 
algorithm. 

We now define what is meant by a clock being nonfaulty in the current frame. The 
predicate nonfaulty.clock is defined as follows: 

Al: Lemma nonfaulty_clock(p, i) = goodclock(p, T^ + Corrp°\T(‘ +1 ) + Corr^) 

8 This differs from the notation, used in [8]. 
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Note that in order for a clock to be non-faulty in the current frame it is necessary that it 
has been working continuously from time 0. 9 

The clock synchronization theory provides two important properties about the clock 
synchronization algorithm, namely that the skew between good clocks is bounded and that 
the correction to a good clock is always bounded. The maximum skew is denoted by 5 and 
the maximum correction is denoted by E. More formally, 

Clock Synchronization Conditions: For all nonfaulty clocks p and q: 

SI: vr € n {i) : | rt®(T) - rt^(T)\ < 6 
S2: |Corrj, ,+1) - Corral < E 

The value of 8 is determined by several key parameters of the synchronization system: 
"rep listed in table 2. The formal definition of p has already been given. The 


parameter 

meaning 

p 

upper bound on drift rate of a good clock 

e 

upper bound on error in reading another processor’s clock 

to 

upper bound on initial skew 

rn 

maximum number of faulty clocks tolerated 

nrep 

number of clocks in system 


Table 2: Meaning of Synchronization Parameters 


parameter e is a bound on the error in reading another processor’s clock. The synchroniza- 
tion algorithm requires that every processor in the system obtain an estimate of its skew 
relative to every other clock in the system. 1 he notation Ag is used to represent the skew 
between clocks q and p during the ith frame as perceived by p. Thus, the real time at which 
p'g clock reads 7 q — should be very close to the real time that q s clock reads Jo. This 
is constrained by an axiom to be less than e: 

Axiom If conditions SI and S2 hold throughout the ith frame, then 
nonfaulty_clock(p,i) A nonfaulty_ciock(r/, i) 

D |Ag| < sync_time 

A ( 3 To : To e S® A |r<j,‘>(7o + Ag) - rfg(T 0 )| < e) 

The amount of time reserved for executing the clock synchronization algorithm is denoted 
by the constant sync_time. 

'Phe third parameter, ^ 0 , i« constrained as follows: 

AO: Axiom |r<^(0) - r<< o) (0)| < /> 0 

9 This is a limitation not of ( lie operating system, but. of existing, mechanically verified fault-tolerant clock 
synchronization theory. Future work will concentrate on how to make clock synchronization robust in the 
presence of transient faults. 
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Thus, 5 0 bounds the initial clock skew. 

The property that the ICA clock synchronization algorithm meets the two synchroniza- 
tion conditions Si and S2 was proved in [8]. These were named Theorem.l and Theorem^: 
formally as: 

Theorem_l: Theorem 

SlA(i) D ( V p, </ : ( V T : 

nonfaulty-dock(p,i) A nonfaulty.clock(</, i) A T G il ( ‘) 

D | rtU(T) - rtW(T) | < 6) 

Theorem_2: Theorem |Corrj,' +,) - Corral < E 
where the premise for Theorem-1, S1A, is defined by: 

(A i : (Vr : (m + 1 <= r and r <= n) D nonfaulty_clock(r,i))) 

and where rn is equal to the maximum number of faulty processors. 

We have used the following equivalent but more convenient premise: SI A : function[period -* 
bool] ==.( A i: enough_clock»(t ) ) . 10 where 

enough-docks: function[period — » bool] = 

( A i : 3 * num_good_clocks(t, nrep) > 2 * nrep) 

and 

num-good_clocks: Recursive functionfperiod, nat — » nat] = 

(A i,k : if k = 0 V k > nrep 
then 0 

elsif nonfaulty-dock(£, i) 

then 1 + num_good_docks(t, k — 1) 
else num_good_clocks(t, k — 1) 
end if) by num.measure 


The theorems proved in [8] also depend upon the following axioms not mentioned above. 
A2.aux: Axiom Ah) = 0 

CO: Axiom m < nrep A m < nrep — num_good_clocks(i, nrep) 

Cl: Axiom frame-time > 3 * sync_time 

C2: Axiom sync.time > E 

C3: Axiom E > A 

C4: Axiom A>/> + < + §* sync_time 

C5: Axiom 6 > S 0 + p * frame-time 

C6: Axiom 6 >2*(e + p* sync.time) + 2 * m * A/(nrep - m) 

-f nrep * p * frame_time/(nrep — rn) + p * A 
-f nrep * p* S/(nrep - m) 

10 Note that this form also subsumes axiom CO below. 
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With the S1A premise expanded, the main synchronization theorem becomes: 

sync.thm: Theorem enough.clocks(i) 

D ( Vp, <jf : (VT : T € A nonfaulty.dock(p, i) A nonfaulty.clock(ij, i) 

D | rt(‘>(T) - rt®(T) \ < 6)) 

The proof that DA implements DS depends crucially upon this theorem. 

7.2 The DA Formalization 

Now that a clock synchronization theory is at our disposal, the DA model can be specified. 

Two new fields are added to the state vector associated with each processor: Iclock and 

cum_delta: 

da.proc_state: Type = Record healthy : nat, 

proc-State : Pstate, 
mailbox : MBvec, 

Iclock : logicaLdocktime, 
cum.delta : number 
end record 


The complete DAstate is: 

DAstate: Type = Record phase : phases, 

sync.period : nat, 
proc : da_proc_array 
end record 

where da.proc-state is defined by: 

da.proc.array: Type = array [processors] of da_proc_state 

The sync.period field holds the current frame of the system. Note this does not represent the 
frame counter on any particular processor, but rather the ideal, unbounded frame counter. 

The Iclock field of a DAstate stores the current value of the processor’s local clock. The 
real-time corresponding to this clock time can be found through use of the auxiliary function 

da_rt. 

da_rt: function[DAstate, processors, logicaLdocktime — ► realtime] = 

( A da,p,T : c P (T + da. proc(p). cum.delta) 

'I'his function corresponds to the rt function of the clock synchronization theory. Thus, 
da_rt(s,p,T) represents processor p’s synchronized clock. Given a clock time T in the current 
frame (s.sync_period), da_rt returns the Teal-time that processor p’s clock reads T . The current 
value of the cumulative correction is stored in the field cum.delta. 

Every frame the clock synchronization algorithm is executed, and is added to cum.delta. 
Note that this corresponds to the Corr function of the clock synchronization theory. The 
relationship between c p , da.rt, and cum.delta is illustrated in figure 9. 
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Figure 9: Relationship between Cp and da_rt 

Since the original ICA clock theory was not cast into the state-machine framework used 
in this work, it is necessary to show that the the da_rt function is equivalent to the rt 
function of the clock synchronization theory. The first step is to equate the period of the 
clock synchronization with the length of a frame in the operating system. Since the length 
of the period in the clock theory is a parameter of the theory, this is accomplished by setting 
it equal to frame-length. Similarly, the execution time of the synchronization algorithm is a 
parameter of the clock theory which is set equal to sync_period. n The clock synchronization 
theory also requires that a constraint be placed on the duration of the sync phase: 

AXIOM: duration(sync) >= sync.period 

The next step is to equate the clocks of the state-machine with the clocks in the sync theory. 
This is done by proving the following lemma: 

da.rt.lem: Lemma reachable(da) A nonfaulty_clock(p, da. sync.period) 

Dda.rt(rf«, P ,r) = rit fasyn ' : - p ' riod, (r) 

This lemma follows from the fact that in every period (during the sync phase) the cum.delta 
field is incremented by A,: 

<.proc(»). cum.delta = s.proc(t). cum.delta + A- s y nc -P er '°d 

The algorithm that is specified in the clock theory uses A; as its correction factor each frame. 
The exact same correction factor is used in the DA model. Thus, the RCP system executes 

11 These are named R and 5 in It), 8]. However, these names conflicted with their use in [1]. 


37 



the same algorithm as specified in the clock theory, and cum.delta will always be equal to 
Corr. Thus, rt v = da_rt p . 

The specification of time-critical behavior in the DA model is accomplished using the 
da_rt function. For example, the broadcast.received function is expressed in terms of da.rt: 


broadcast-received: function[DAstate, DAstate, processors -+ bool] = 

: ( Vp : 

(s.proc(p). healthy > 0 

A da_rt(.s, p, .s.proc(p).ldock) + max.comm_delay 
< da_rt(<, <7, t.proc(</).lclock) 

3 t.proc(</).mailbox(p) = s.proc(p).mailbox(p) 

Thus, the data in the incoming bin p on processor q is only defined to be equal to the value 
broadcast by p (i.e. s.proc(p).mailbox(p)) when the real time on the receiving end (i.e. 
da-rt(t,<7,*.proc(<7).lclock) is greater than da-rt(s,p,s.proc(p).ldock) plus max.comm-delay. 
This specification anticipates the design of a communications system that can deliver a 
message in a bounded amount of time, in particular within max_comm_delay units of time. 

In the DA level there is no single transition that covers the entire frame. There is only a 
transition relation for a phase. The Mda relation is. 


Mda- function[DAstate, DAstate, inputs -+ bool] = 

( \ s , : enough-hardware(f) A t. phase = next_phase(s. phase) 

A (Vi : if s. phase = sync 
then •A/’j’oCMj*) 

else t.proc(r). healthy = s.proc(r). healthy 

A <.proc(i).cum_delta = s.proc(t).cum_deIta 
A <.sync_period = s.sync_period 
A (nonfaulty_clock(i,.s.sync_period) 

3 dock_advanced(s.proc(i).ldock, t ,proc(i).lclock, duration(s. phase))) 
A (s. phase =. compute 3 Md a (s,t,u,i)) 

A (s. phase = broadcast 3 -A /J a 0M»O) 

A (s. phase = vote 3 
end if)) 

Note that the transition to a new state is only valid when the enough-hardware function holds 
in the next state. This function is defined as follows. 

enough-hardware: function [DAstate — ► bool] = 

(A t : maj-working(^) A enough_clocks(£.sync_period)) 

maj-working is defined identically in RS. DS. and DA. Its definition is presented in section 3.3. 

The definition of enough-docks appears in section 7.1. 

As in the DS level, the state transition relation Mda is defined in terms of four sub- 
relations, each of which applies to a particular phase type. These are called M c ia , M da , 
and M da - 

The sub-relation is: 
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N% a . function[DAstate, DAstate, inputs, processors — ♦ bool] = 

( A s,t,u,i : 

s.proc(i). healthy > 0 

3 J.proc(i).proc.state = / r («,s.proc(i).proc.state) 

A <.proc(i).mailbox(?:) = /,(/ r (u,.s.proc(/) proc-state)) 

Just as in the corresponding DS relation, the proc_state field is updated with the results of 
the computation, / c (u, .s.proc(i).proc-state). Also, the mailbox is loaded with the subset of 
the results to be broadcast as defined by the function /,. Unlike the DS model, the local 
dock time is changed in the new state. This is accomplished by the predicate clock-advanced, 
which is not based on a simple incrementation operation because the number of clock cycles 
consumed by an instruction stream will exhibit a small amount of variation on real processors. 
The function dock-advanced accounts for this variability, meaning the start of the next phase 
is riot deterministically related to the start time of the current phase. 

w. number 

clock-advanced: function [logicaLclocktime, logicaLclocktime, number -» bool] = 

( A X, Y, D : X + D * (1 - u) < Y A V' < X + D * (1 + u)) 

where v represents the maximum rate at which one processor’s execution time over a phase 
can vary from the nominal amount given by the duration function, // is intended to be a 
nonnegative fractional value, 0 < o < 1. The nominal amount of time spent ill each phase 
is specified by a function named duration: 

duration: function[phases — + logicaLclocktime] 

However, the actual amount of clock time spent in a phase is not fixed, but can vary within 
limits. For example, the actual duration of the compute phase can be anything from (1 - 
i/) * duration(compute) to (1 + u) * duration(compute). The value of v is a parameter of the 
specification and can be set to any desired value. However, there are some constraints on 
the implementation that are expressed in terms of v\ 

broadcast-duration: Axiom 

duration (broadcast)* (l-P-2* //♦duration (compute)— //*duration(broadcast))— 

8 > max-Comm_delay 

broadcast_duration2: Axiom 

duration(broadcast) - 2 * v * duration(compute) - v * duration(broadcast) >= 0 
pos.durations: Axiom 

0 <= (1 — v) * duration(compute) A 0 <= (1 — */) * duration(broadcast) 

A 0 <= (1 - u) * duration(vote) A 0 <= (1 — u) * duration(sync) 

all-durations: Axiom 

(1 + //) * duration(compute) + (1 + //) * duration(broadcast) 

< frame-time 
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The constants p and S are drawn from the clock synchronization theory, as explained in 

section 7.1. ..... ,./r 

There may be many possible causes of the variation in execution times on different 

processors. The asynchronous interface between a processor and its memory can lead to 
different execution times between two processors even when they execute exactly the same 
instructions on exactly the same data. Another possible cause of different execution times 
could be the use of different schedules on different processors. 

The jVX sub-relation is: 

A/J 0 : fu notion [DAstate,DAstate, processors -> bool] = 

(A s,t,i : s.proc(r). healthy > 0 

3 £.proc(i).proc_state = s.proc(i).proc_state 

A broadcast.received(s,i,i)) 

As in the corresponding DS relation, the proc.state field remains unchanged and the broad- 
cast_received relation must hold. When it holds, all the nonfaulty processors receive the 
values sent by other nonfaulty processors. However, this is now contingent upon certain 
constraints on the times that things happen. 

Th eA/j„ sub-relation is: 

Wj o : function[DAstate, DAstate, processors — > bool] = 

(As, t,i : s.proc(i). healthy > 0 

3 <.proc(i). mailbox = s.proc(i). mailbox 

A t.proc(?').proc-state = /„(s.proc(i).proc_state, s.proc(i). mailbox)) 

As before, the mailbox field remains unchanged and the local processor state is updated with 
the result of voting the values broadcast by the other processors. 

The A/j 0 sub-relation is: 

Wj a : function[DAstate, DAstate, processors — » bool] = 

(A s,t,i : (s.proc(f). healthy > 0 

3 Uproc(i).procjstate = s.proc(i).proc-state) 

A (Lproc(i). healthy > 0 

3 Aproc(i). healthy = 1 + s.proc(i). healthy 
A nonfaulty.clock(?', t.sync_period)) 

A £.sync_period = 1 + s.sync.period 
A (nonfaulty_dock(i, s.sync_period) 

3 Aproc(i).lclock = (1 + s.sync.period) *frame_time 

AAproc(i).cum_delta = s.proc^j.cum.delta+A; )) 

During the sync phase, the processor state remains unchanged. As in the DS specification, 
the healthy field is incremented by one. Unlike the DS model, the local clock time is changed 
in the new state. For this sub-relation, the clock is not advanced in accordance with the 
function clock-advanced, because this phase is terminated by a clock interrupt. At a pre- 
determined local clock time, the clock interrupt fires and the next frame is initiated. The 
specification requires that the interrupts fire at clock times that are integral multiples of the 
frame length, frame_time. 
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In addition to requirements conditioned on having a nonfaulty processor, the DA speci- 
fications are concerned with having a nonfaulty clock as well. It is assumed that the clock 
is an independent piece of hardware whose faults can be isolated from those of the corre- 
sponding processor. Although some implementations of a fault-tolerant architecture such as 
RCP could execute part of the clock synchronization function in software, thereby making 
clock faults and processor faults mutually dependent, we assume that RCP implementations 
will have a dedicated hardware clock synchronization function. This means that a clock can 
continue to function properly during a transient fault period on its adjoining processor. The 
converse is not true, however. Since the software executing on a processor depends on the 
clock to properly schedule events, a nonfaulty processor having a faulty clock may produce 
errors. Therefore, a one-way fault dependency exists. 
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Figure 10: Relationship of clock and processor faults. 

Figure 10 summarizes the interaction between clock faults and processor faults. It shows 
for each combination of fault mode whether a processor can make a sound contribution to 
voting the state variables and whether a clock can properly contribute to clock synchroniza- 
tion. These conditions have been encoded in the various DA specifications. In particular, 
the relation shown above requires that for a processor to be nonfaulty in the next frame 
it must have a nonfaulty clock through the end of that frame. Recall that the definition of 
nonfaulty clock requires that it be continuously nonfaulty from time zero. 12 

The predicate initial.da puts forth the conditions for a valid initial state. The initial phase 
is set to compute and the initial sync period is set to zero. Each element of the DA state array 
has its healthy field equal to recovery.period and its proc_state field equal to initial_proc_state. 

initial.da: function [DAstate — » bool] = 

(A s : s. phase = compute A s.sync.period = 0 

A ( Vi : s.proc(i). healthy = recovery.period 

A s.proc(i).proc_state = initial_proc_state 
A .s.proc(*).cum.deIta = 0 
A s.proc(i).lclock = 0 A nonfaulty_dock(i, 0))) 

As before, the constant recovery.period is the number of frames required to fully recover a 
processor’s state after experiencing a transient fault. By initializing the healthy fields to this 

12 This does not represent a deficiency in the design of the DA model but rather is a limitation imposed 
by the existing, mechanically verified clock synchronization algorithm. Future work will concentrate on 
liberating the clock synchronization property from this restriction. 
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value, we are starting the system with all processors working. Note that the mailbox fields 
are not initialized; any mailbox values can appear in a valid initial DAstate. 

8 DA to DS Proof 

8.1 DA to DS Mapping 

The DA to DS mapping function, DAmap, is defined as: 

DAmap: function[DAstate — + DSstate] = 

(A da : ss_update(da, nrep) with [(phase) := da. phase]) 

where ss.update is given by: 

ss-update: Recursive function[DAstate, nat — * DSstate] = 

( A da, I k : if (k = 0) V (k > nrep) 
then dsO 

else ss_update(da, k — 1) 
with [(proc)(fc) := dsprocO 

with [(healthy) := da. proc( k). healthy, 

(proc_state) := da.proc(fc).proc_state, 

(mailbox) := da. proc(fc). mailbox]] 
end if) by da.measure 

Thus, the Iclock, cum.delta, and sync.period fields are not mapped (i.e., are abstracted away) 
and all of the other fields are mapped identically. To establish that DA implements DS, 
the commutativity diagram of figure 11 must be shown to commute. To establish that the 
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Figure 1 1: Commutative Diagram for DA to DS 1 roof 

diagram commutes, the following formulas must be proved. 

phase-commutes: Theorem reachable(s)AjVd u (s, t, u) D Afds{ DAmap(s), DAmap(t), u) 
initiaLmaps: Theorem initiaLda(s) D initial_ds(DAmap(.s)) 
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The lemmas below directly follow from the definition of the mapping. 

map-1: Lemma DAmap(.s).proc(r). healthy = .s.proc(i). healthy 
map_2: Lemma DAmap(s).proc(i).procj>tate = s.proc(i).proc_state 
map_3: Lemma DAmap(.s). phase = s. phase 
map-4: Lemma DAmap(s).proc(i). mailbox = s.proc(i). mailbox 
map_7: Lemma DS.maj_working(DAmap(s)) = DA.maj.working(s) 

8.2 The Proof 

The phase.com mutes theorem must be shown to hold for all four phases. Thus, the proof is 
decomposed into four separate cases, each of which is handled by a lemma of the form. 

phase.com..Y: Lemma 

s. phase = X A Mda{s, t, u) 3 Afd,{ DAmap(s), DAmap(*), «) 

where X is any one of {compute, broadcast, vote, sync}. The proof of this theorem requires 
the expansion of the Af da relation and showing that the resulting formula logically implies 
A/j,(OAmap(s), DAmap(i), u). 

8.2.1 Decomposition Scheme 

The proof of each lemma phase_com_,Y is facilitated by using a common, general scheme for 
each phase that further decomposes the proof by means of four subordinate lemmas. The 
general form of these lemmas is as follows: 

Lemma 1: s . phase = X AA/ia(s,bw) 3 (Vi : Afj£(.s,t,i)) 

Lemma 2: s . phase = X A Af? a {s,t,i) 3 A/£'(DAmap(s), DAmap(t), i) 

Lemma 3: s. phase = X A DS.maj.working(M) A (Vi : Af£(ss,tt,i)) 3 
•X’dai.SS , tt , 

Lemma 4: s. phase = .V A Af da (s,t,u) 3 DS.maj_working(DAmap(t)) 

A few differences exist among the lemmas for the four phases, but they adhere to this scheme 
fairly closely. The phase.com., Y lemma follows by chaining the four lemmas together: 

( V i : A/£(DAmap(.s), DAmap(l), *)) 3 A/; is (DAmap(s), DAmap(0, u) 

In three of the four cases above, proofs for the lemmas are elementary. The proof of 
Lemma 1 follows directly from the definition of Af da . Lemma 3 follows directly from the 
definition of Af dt . Lemma 4 follows from the definition of N da , enough-hardware and the basic 

mapping lemmas. 

Futhermore, in three of the four phases, the proof of Lemma 2 is straightforward. For 
all but the broadcast phase, Lemma 2 follows from the definition of M da , A f da , and the basic 
mapping lemmas. 
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However, in the broadcast phase, Lemma 2 from the scheme above, which is named 
com broadcast-2, is a much deeper theorem. The broadcast phase is where the effects of 
asynchrony are felt; we must show that intcrproccssor communications are properly received 
in the presence of asynchronously operating processors. Without clock synchronization we 
would be unable to assert that broadcast data is received. Hence the need to invoke clock 
synchronization theory and its attendant reasoning over inequalities of time. 

8.2.2 Proof of com_broadcast_2 

The lemma com-broadcast_2 is the most difficult of the four lemmas for the broadcast phase. 
It follows from the definition of A&, W] u , the basic mapping lemmas and a fairly difficult 
lemma, com_broadcast_5: 

corri-broadcast_5: Lemma 

reachable(s) A A^ a (s,<,u) A s. phase = broadcast 
A s.proc(i). healthy > 0 A broadcast_received(.s, f, i) 

3 broadcast-received(DAmap(s), DAmap(J),r) 

This lemma deals with the main difference between the DA level and the DS level the 
timing constraint on the function broadcast-received: 

broadcast-received: function[DAstate, DAstate, processors — ► bool] 

(A s,t,q : (Vp : 

(s.proc(p). healthy >0 

Ada_rt(s,p, (s.proc(p).lclock)+max_comm_deIay < da_rt(J, 9 ,Tproc(<j).lclock) 

3 <.proc(< 7 ).mailbox(p) = s.proc(p).mailbox(p) 

The timing constraint 

da_rt(s, p, s.proc(p).lclock) + max_comm_delay < da_rt(f,g,t.proc(< 7 ).ldock) 

must be discharged in order to show that the DA level implements the DS level. The following 
lemma is instrumental to this goal. 

ELT: Lemma T 2 > 7', + bb A (7’, > T°) A (bb > T°) A T 2 6 R {sp) A 7\ € R (s p) 

A nonfaulty_clock(p, sp) A nonfaulty_dock(r/,sp) A enough_clocks(sp) 

3 rtf P) (T 2 ) > r4 SP) ('A) + (1 - f) * |bb] - S 

This lemma establishes an important property of timed events in the presence of a fault- 
tolerant clock synchronization algorithm and is proved in the next subsection. Suppose that 
on processor q an event occurs at 7T according to its own clock and another event occurs on 
processor p at time T2 according to its own clock. Then, assuming that the clock times fall 
within the current frame and the clocks are working and the system still is safe (i.e. more 
than two thirds of the clocks arc non-faulty), then the following is true about the real times 

of the events: 

r4 Sp) (7 2 ) > r4 SP) (7’.) + (1 - f) * |bb| - 6 
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where bb = T 2 - T u T, = s.proc(p).ldock and 7’ 2 = i.proc( 9 ).ldock. 

If we apply this lemma to the broadcast phase, letting Tl be the time that the sender 
loads his outgoing mailbox bin and T2 is the earliest time that the receivers can read their 
mailboxes (i.e. at the start of the vote phase), wo know that these events are separated in 

time by more than (1 — §) * |bb| — 8. 

In this case bb is approximately equal to duration(broadcast). However, since theie may 
be some variations in the time spent in the compute and broadcast phases on different 
processors (i.e. they can drift from the nominal value at a rate less than v ) , the analysis is a 
little tricky. First consider the situation where processor <7 is sending a message to processor 
p during its broadcast phase. Let r be the state at the start of the compute phase, s be the 
state at the start of the broadcast phase and t be the state at the start of the vote phase: 

compute broadcast . 

r — -> S > t 

Then, let 

Rq = the clock time at the start of the compute phase on processor q 

Sq = the clock time at the start of the broadcast phase on processor q 

Tq = the clock time at the start of the vote phase on processor q 

Rp = the clock time at the start of the compute phase on processor p 

Sp r= the clock time at the start of the broadcast phase on processor p 

Tp = the clock time at the start of the vote phase on processor p 

This is illustrated in figure 12. By the definition of clock_advanced, the following can be 
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Figure 12: Relationship between phase times on different processors 


established: 

( 3 pdurc, pdurb, qdurc, qdurb : 

near(pdurc, compute) A near(pdurb, broadcast) 

A near(qdurc, compute) A near(qdurb, broadcast) 

A Rp = Rq 
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A Sq = Rq + qdurc 

A Tp = Sq - qdurc + pdurc + pdurb)) 
where near(dur.ph) is given by 

near(dur.ph) =(]-«/)* duration(ph) < dur < (I + /') * duration(ph)) 

This result depends upon a critical invariant of the system: 

( V p, <7 : s. phase = compute A 

nonfaulty_clock(p, s.sync_period) A nonfaulty_clock(7,s.sync_period) 

3 s.proc(p).lclock = s.proc(r/).lclock) 

given that the state s is reachable(s). This invariant exists in the system because of the 
use of an interrupt timer to initiate the start of a frame on each of the processors at the 
pre-determined times i *frame_time. Using the definition of R ^ and the axioms pos_durations 
and all-durations, we obtain: 

nonfaulty_clock(p, r) A nonfaulty.clock(</, i) 

3 Sq £ R^ A Tp € R^ 

A Tp > Sq + duration(broadcast) 

— 2 * v * duration (compute) — u * duration(broadcast) 

where i is the current synchronization period (i.e. i = r.sync_period = s.sync.period 
l. sync-period). We now have a relationship between the clock time that the message was 
sent and the clock time that it was received in a form appropriate for application of the ELT 
theorem. In other words, Ti = Tp, 'I\ = Sq and bb = pdurc — qdurc + pdurb. Thus, we 
can convert the relationship between the events expressed in clock times to a relationship 
between the real times of these events: 

r*h‘)(Tp) > rt^{ Sq) + (1 - § ) * |duration(broadcast) - Epsi| - 6 

where Epsi = 2 * v * duration(compute) + v *duration(broadcast). Using the broadcast-duration 
implementation axiom: 

broadcast_duration: Axiom 

duration (broadcast) * (\ - %) - 2 * v * duration(compute) 

- v* duration(broadcast)) - 6 > max-comm-delay 

we have: 

ri£' J (T p) > r^'l(Sq) + max_comm_delay 
Using the da.rtJem lemma: 

da.rt(t, q, Tq) >= rfa_r<(.*»,p, Sq) + max-comm.delay 
This will discharge the premise of broadcast, received. Thus, 
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com.broadcastJj: Lemma 

reachable(s) A Afda( s , L ?1 ) A s. phase = broadcast 
A s.proc(p). healthy > 0 A broadcast.received(s, t,p) 

3 broadcast-received(DAmap(s), DAmap (f),p) 

Of course there are several technicalities such as the reachable(da) premise that must be dis- 
charged in order to apply the da.rt Jem lemma and the other slate invariants and establishing 
that s.proc(p). healthy >03 nonfaulty_clock(p, ,s.sync_period). 

Proof of ELT Lemma: In this section we prove, 

Lemma 1 (earliest_later_time Lemma) T 3 = T x + BB 

A (T| > 7’°) A (BB > T°) A nonfaulty_clock(p, i) A nonfaulty_clock(< 7 ,z) 

A enough_clocks(i) A T 3 € R^ A T\ € R^ 

3 rtW{Ti) > riW(Ti) + (1 - f) * |BB| - 8 
from which the ELT lemma immediately follows. 

Proof. This lemma depends primarily upon the definition of a good clock and the synchro- 
nization theorem (i.e. sync_thm). The good clock definition yields: 

goodclock(<3 i , T°, Tj + BB) A (T\ > T°) A (BB > 7°) 

3 (1 — f) * I BB | < c,(Tj + BB) - c q (7\) 

A C ,(T 1 + BB)-c,(T 1 )<(l + f)HBB| 

Note that the definition of a good clock is defined in terms of the uncorrected clocks, c p (T). 
Using the definition of rt, we can rewrite the first formula as: 

Lemma goodclock(< 7 ,'f°, T\ + Corr ^ -1- BB) 

A (Ti > T°) A (T, + CW« > T°) A (BB > T°) 

3 (1 - f) * |BB| < rlf)(T, + BB) - 
A riW(Ti + BB) - r<W(r,) < (I + f) * |BB| 

and obtain a formula in terms of the function rt. 

The sync.thm theorem gives us: 

enough.clocks(t) A nonfaulty_clock(/>, i) A nonfaulty_clock(< 7 , i) A T € 

3 -6 < r tf(T) - vlf(T) < « 

Combining the previous two formulas and substituting F 2 for T in sync_thm, we obtain. 

T 2 = T, 4- BB A (T, > T°) A (T, + Corr}') > T°) A (BB > T°) A T 2 € 7? (,) 

Aenough_clocks(i)Agooddock(</, 7 l0 , 7\C'orr^+BB)Anonfaulty_dock(p, t)A 
nonfaulty_clock(( 7 , i) 

3 W<‘)(7 2 ) > H<‘>(T,) + (1 - f) * |BB| - 5 

From the definition of nonfaulty and goodclock, we have: 

7\ + BB < T<'' +l > A nonfaulty-dock(r/, /) 

3 goodclock(</, 7’°, T, + Corr<’> + BB) 
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Using these last two results we have: 

T 2 = T x + BB A T 2 < A (T, > T°) A (7\ + Corrf) > T °) A (BB > T°) 

A enough.clocks(i) A nonfaulty_clock(p, i) A nonfaulty_clock(<jr,z) A T 2 € R (,) 

3 r/W(T 2 ) > rl»(T,) + (1 - f) * |BB| - 5 

Then from the definition of 7' (,) and the fact that Corr<°> = 0, we have 

ftll: Lemma T 2 = 7^ + BB A (7'i > 7'°) A (7’, + > 7 °) A (BB > T°) 

A enough clocks(i) A nonfaulty_clock(p, i) A nonfaulty_clock(<z,z) A T 2 <E 7? ( ') 
3H(;)(T 2 )>rf(d(7\) + (l-f)HBB|-^ 

Using the adj.always.pos theorem from [8], we obtain 

ftl2: Lemma T\ € 3 (7\ + Corr^ > T°) 

The key lemma follows immediately from the last two formulas, (ftll and ftl2). 

9 Implementation Considerations 

Although many liCP design decisions have yet to be made, there are a number of implemen- 
tation issues that need to be considered early. Some of these have emerged as consequences 
of the formalization effort completed in Phase 2. Others are the result of preliminary investi- 
gations into the needs of implementations that can satisfy the RCP specifications. Following 
is a discussion of these issues and available options. 

9.1 Restrictions Imposed by the DA Model 

Recall that the DA extended state machine model described in section 2.4 recognized four 
different classes of state transition: L, R, R, C. Although each is used for a different phase 
of the frame, the transition types were introduced because operation restrictions must be 
imposed on implementations to correctly realize the DA specifications. Failure to satisfy 
these restrictions can render an implementation at odds with the underlying execution model, 
where shared data objects are subject to the problems of concurrency. The set of constraints 
on the DA model’s implementation concerns possible concurrent accesses to the mailboxes. 

While a broadcast send operation is in progress, the receivers’ mailbox values are unde- 
fined. If the operation is allowed sufficient time to complete, the mailbox values will match 
the original values sent. If insufficient time is allowed, or a broadcast operation is begun 
immediately following the current one, the final mailbox value cannot be assured. Further- 
more, we make the additional restriction that all other uses of the mailbox be limited to 
read-only accesses. This provides a simple sufficient condition for noninterfering use of the 
mailboxes, thereby avoiding more complex mutual exclusion restrictions. 

Operation Restrictions. Let s and t be successive DA states, i be the proces- 
sor with the earliest value of c,-(s(i). Iclock), and j be the processor with the latest 
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value of Cj(t(j). Iclock). If s corresponds to a broadcast (B) operation, all proces- 
sors must have completed the previous operation of type R by time c,(s(i). Iclock), 
and the next operation of type B can begin no earlier than time Cj(t{j). Iclock). 

No processor may write to its mailbox during an operation of type B or R. 

By introducing a prescribed discipline on the use of mailboxes, we ensure that the axiom 
describing the net effect of broadcast communication can be legitimately used in the DA 
proof. Although the restrictions are expressed in terms of real time inequalities over all 
processors’ clocks, it is possible to derive sufficient conditions that satisfy the restiictions and 
can be established from local processor specifications only, assuming a clock synchronization 
mechanism is in place. 


9.2 Processor Scheduling 

The DA model of the RCP deals with the timing and coordination of the replicated processors 
in a fairly complete manner. The model defines in detail the functionality of the system with 
regard to the activities that are necessary to ensure its fault- masking and transient recovery 
capability. Nevertheless, the delineation of the task execution process on each local processor 
has not been elaborated in any more detail than in the US model. This was done deliberately 
in order to obtain as general a specification as possible. Thus, the 4-level hierarchy presented 
in this paper could be further refined into a set of entirely different kinds of implementations. 
They could differ drastically in the types of task scheduling that are utilized as well as the 
type of hardware or software used. 

Nevertheless, one aspect of scheduling needs to be carefully controlled, namely the basic 
frame structure. The RCP specifications were developed with a very crisp execution model 
in mind regarding the basic timing of a frame and its major parts. We assume the existence 
of one or more nonmaskable hardware interrupts, triggered by the clock subsystem, that are 
used to effect the transition from one frame to the next and one major phase to the next. 
As a minimum, the following transitions must be triggered by timer interrupts or an equally 
strong hardware mechanism. 

• Start of frame. The last portion of a frame is reserved for clock synchronization 
activities. This includes not only executing the clock synchronization functions, but 
also reserving some dead time to be sacrificed when clock adjustments cause local clock 
time discontinuities. An interrupt is set to fire at the proper value of clock time so 
that all processors begin the new frame with the same local clock reading. 

• Beginning of vote phase. After waiting for the completion of broadcast communi- 
cation from other processors, the vote phase is begun to selectively restore portions of 
the computation state. Also needing to be recovered are any control state variables 
used by the operating system. If a transient fault occurs, recovery cannot begin until 
the control state is first restored through voting. However, a processor operating after 
a transient fault may be executing with a corrupted memory state. The only way to 
ensure that corrupted memory does not prevent the eventual recovery of control state 
information is to force the vote to happen through a nonmaskable interrupt. 
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The use of timer interrupts arc highly desirable in other situations, but those listed above 
are considered essential. 

Scheduling of applications tasks is an area where the implementation retains some flex- 
ibility owing to our use of a general fault-tolerant computing model in the US and RS 
specifications. Often it is considered desirable to achieve some type of schedule diversity 
across processors as a means of gaining more transient fault immunity. A limited way of 
accomplishing this is available under the current RCP design. Since the specifications only 
state what must be true after all tasks have been executed within a frame, it is possible to 
juggle the order of tasks within each frame to implement diversity. For example, if N tasks 
are scheduled in a particular frame, each processor may execute them in a different order 
up to the limits of data dependency among tasks. It is also possible to introduce different 
spreads of slack time, dummy tasks, etc. to achieve similar effects. 

9.3 Hardware Protection Features 

Correct recovery of state information after a transient fault has been formalized in the RS 
to US proof. Transient recovery of state information occurs gradually, one cell at a time. 
Consequently, depending on the voting pattern used, some tasks will be executing in the 
presence of erroneous state information. Implicit in the RS specifications is that computation 
of task outputs is not subject to interference by other tasks executing with erroneous data 
inputs. In the specifications, this is due simply to the use of a functional representation of 
the eirects of task execution. 

Nonetheless, in a real processor a program in execution can interfere with another unless 
hardware protection mechanisms are in place. To see why this is so, suppose, for instance, 
that task 7j is followed by task l\ in a particular frame and neither s output is voted during 
that frame. Suppose further that in the transient fault recovery scheme, T 2 ' s inputs come 
from recently voted cells while T,’ s do not. Thus, wc expect 7Vs cell to be recovered after this 
frame. After a transient fault, 7 1 , may be executing instructions on erroneous data, possibly 
overwriting recovered information such as that required by T 2 . This would invalidate our 
assumption that T 2 's state is recovered at the end of the frame. 

In a similar manner, interference can be caused in the time domain as well as the data 
domain. In the example above, if T,’s erroneous input causes it to run longer than its upper 
execution time bound, T 2 may not get to execute in this frame. Again, this would result in 
our assumptions about T 2 ’s output being invalid. Therefore, hardware protection features 
are required to prevent both kinds of interference in a system that attempts to recover state 
information selectively. 

There are several well-known hardware techniques for providing this type of protection. 

• Memory protection. Hardware write protection devices are found on many modern 
computer architectures. What RCP requires is less than a full-blown memory manage- 
ment, unit (MMU). All that is necessary is to be able to prevent a task in execution 
from writing into memory areas for which the operating system has not given explicit 
write permission. The ability to give a task write access to a small set of physical 
memory regions is sufficient. Generating hardware exceptions such as traps on illicit 
write attempts is desirable but not essential. 
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• Watch-dog timers. Timer interrupts or special-purpose timing logic will be required 
to prevent a task from consuming more than its allotted amount of execution time. 
When a watch-dog timer is triggered, the operating system need only dispatch the 
next task on the schedule. The actual hardware used to carry out this timing function 
needs to have adequate resolution and be distinct from the timer interrupts used to 
signal the end-of-frame and start-of-voting events. 

• Privileged Operating Modes. To protect the protection mechanisms, it is usually 
necessary for a processor to have at least one privileged execution domain. Processors 
typically provide at least a user domain and a (privileged) supervisor domain to im- 
plement conventional operating system designs. In RCP, we need these features so the 
tasks cannot accidentally change or disable the memory write protection or watch-dog 
timer functions. There may be other uses for privileged mode as well. 

It is important to realize that use of these features may be obviated in special cases. If 
sufficiently frequent voting is used, for example, it may not be necessary to provide these 
features as long as a task is always executing with valid data as input. 

9.4 Voting Mechanisms 

Exact-match voting of state information exchanged among processors is usually envisioned 
as applying the majority function to mailbox values. Note, however, that the voting function 
/„, described in section 3.3, is unspecified and need not be based on the majority operation. 
Other types of voting may be used provided that the transient recovery axioms of section 3.5.2 
are still true. 

A desirable alternative to majority voting is plurality voting. If the values subject to 
voting are {a, a, 6, c}, for example, a majority does not exist, but a plurality does, namely 
{a, a}. The reason this can be valuable is that during a massive transient fault that affects 
more than a majority of processors, the Maximum Fault Assumption no longer holds and 
transient fault recovery is not assured by the proofs previously described. However, the 
likelihood is that the affected processors will not exhibit exactly the same errors. If a 
minority of processors is still working, it is likely that the values produced by the replicated 
processors will appear something like the example {a, a, 6, c}. Hence, plurality voting has a 
good chance of recovering the correct state in spite of the absence of a working majority. 

This problem has been studied by Miner and Caldwell [26]. They showed that the 
substitution of plurality voting for majority voting can be used to produce identical results 
as long as the Maximum Fault Assumption holds: 

maj_exists(s) D maj(s) = plur(s) 

By using an implementation based on plurality voting, we enjoy the same provable behavior 
when the Maximum Fault Assumption holds, and we enjoy added transient fault immunity 
in the rare case that it is violated. All that is necessary to achieve this is to show that the 
choice of function for /„ meets the requirements of the transient recovery axioms. 
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10 Future Work 

There are four main areas where further work may be profitable. 

1. Development of a still more detailed specification and verification that it meets the DA 
specification. 

2. Development of task scheduling/voting strategies that satisfy the axioms of the US 
model. 

3. More detailed specification of the behavior of the actuator outputs. 

4. Development of a detailed reliability model. 

10.1 Further Refinement 

Although the DA specification is a fairly detailed design of the system- wide behavior of the 
RCP, there is very little implementation detail about what occurs locally on each processor. 
The next level of the specification hierarchy, the local processor LP specification will define 
the data structures and algorithms to be implemented on each local processor. 

At some point the design must be implemented on hardware. It is anticipated that both 
standard hardware such as microprocessors and memory management units will be required 
as well as special hardware to implement the clock synchronization and Byzantine agreement 
functions. In the same way that this work capitalized on the work done elsewhere in clock 
synchronization, the LP specification will build on the work being performed under contract 
to NASA Langley in hardware verification. 

NASA Langley has awarded three contracts specifically devoted to formal methods (from 
the competitive NASA RFP 1-22-9130.0238). The selected contractors were SRI Interna- 
tional, Computational Logic Inc., and Odyssey Research Associates. Another task-assignment 
contract with Boeing Military Aircraft Company (BMAC) is being used to explore formal 
methods as well. Through this contract BMAC is funding research at the University of 
California at Davis and California Polytechnic State University to assist them in the use of 
formal methods in aerospace applications. The efforts arc roughly divided as follows: 

SRI: Clock synchronization, operating system 

CLI: Byzantine Agreement Circuits, clock synchronization 

ORA: Byzantine Agreement Circuits, applications 

BMAC: Hardware Verification, formal requirements analysis 

The DA specification critically depended upon a clock synchronization property. Previous 
work by SRI had verified that the ICA algorithm meets this property. Ongoing work at SRI 
is directed at implementing a synchronization algorithm in hardware verifying it. This will 
lead to the verification hierarchy shown in figure 13. 

Implicit in the RS, DS and DA models is the assumption that it is possible to distribute 
single source information such as sensor data to the redundant processors in a consistent man- 
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Figure 13: Clock Synchronization Hierarchy 

ner even in the presence of faults. This is the classic Byzantine Generals problem [18]. 13 CLI 
is investigating the formal verification of such algorithms and their implementation. They 
have formally verified the original Pease, Shostak, and Lamport version of this algorithm 
using the Boyer Moore theorem prover [27]. They have also implemented this algorithm 
down to the register-transfer level and demonstrated that it implements the mathematical 
algorithm [28]. Future work will concentrate on tying this work together with their verified 
microprocessor, the FM8502 [29]. 

ORA has also been investigating the formal verification of Byzantine Generals algorithms. 
They have focused on the practical implementation of a Byzantine-resilient communications 
mechanism between Mini-Cayuga micro-processors [30]. The Mim-Cayuga is a small but 
formally verified microprocessor developed by ORA. It is a research prototype and has not 
been fabricated. This communications circuitry could serve as a foundation for the RCP 
architecture. It was designed assuming that the underlying processors were synchronized 
(say by a clock synchronization circuit). The issues involved with connecting the Byzantine 
communications circuit with a clock synchronization circuit and verifying the combination 

have not yet been explored. mao a 

Boeing Military Aircraft Company and U. C. Davis have been sponsored by JNAbA, 
Langley to apply formal methods to the design of conventional hardware devices. Formal 
Verification of the following circuits is currently under investigation: 

• a floating-point coprocessor similar to the Intel 8087 (but smaller) [31, 32]. 

• a DMA controller similar to the Intel 8237A (but smaller) [33]. 

• microprocessors in IlOL (small) [31, 35, 36]. 

• a memory management unit [37, 38]. 

13 Pault- tolerant systems, although internally redundant, must deal with single-source information from 
the external world. For example, a flight control system is built around the notion of feedback from physical 
sensors such as accelerometers, position sensors, pressure sensors, etc. Although these can be replicated 
(and they usually are), the replicates do not produce identical results. In order to use bit-by-bit majority 
voting all of the computational replicates must operate oil identical input data. Thus, the sensor values (t e 
complete redundant suite) must be distributed to each processor in a manner that guarantees all working 
processors receive exactly the same value even in the presence of some faulty processors. 
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The team is currently investigating the verification of a composed set of verified hardware 
devices [39, 40, 41] 

Researchers at NASA Langley have begun a new effort on a hardware clock synchro- 
nization technique that can serve as a foundation for the RCP architecture. The method, 
which is based on the Fault-Tolerant Midpoint algorithm [42], is aimed at a fully independent 
hardware implementation. The primary goals of this work are full mechanical verification, 
transient fault recovery, and an initialization scheme that provides recovery from large tran 
sient upsets. 

10.2 Task Scheduling and Voting 

The Phase 1 report described a scheduling system that was based upon a deterministic table. 
In the models presented in this paper, this is no longer strictly required although such an 
approach clearly fits within the axioms presented in the US model. However, it is conceivable 
that more sophisticated scheduling strategies could also be shown to conform. 


10.3 Actuator Outputs 

It is important not only that the replicated outputs sent to the actuators [on separate wires) 
are identical but that they appear within some bounded time of each other. Although this 
bound may not be very small, it is still incumbent upon the verification activity that a bound 
be mathematically established. 

10.4 Development of a Detailed Reliability Model 

In the Phase 1 paper, a simple reliability model of the RCP system was developed that 
demonstrated that the speed at which one must remove the effects of a transient fault is 
not very critical. In other words, flushing the effects of a transient fault over an extended 
period of time did not significantly decrease the reliability of the system as compared to 
extremely fast removal. In this model, a fault anywhere in the processor was sufficient to 
render the entire processor faulty. Clearly, in a fully developed RCP, there will be more 
than one fault-isolation containment region per processor. The most likely arrangement is to 
have a separate fault-containment region for the clocking system and one for the Byzantine 
agreement circuitry. 


11 Concluding Remarks 

In this paper a hierarchical specification of a reliable computing platform (RCP) has been 
developed. The top level specification is extremely general and should serve as a model for 
many fault-tolerant system designs. The successive refinements in the lower levels of abstrac- 
tion introduce, first, processor replication and voting, second interprocess communication by 
use of dedicated mailboxes and finally, the asynchrony due to separate clocks in the system. 
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Although the first phase of this work was accomplished without the use of an automated 
theorem prover, we found the use of the Ell DM system to he beneficial to this second phase 
of work for several reasons. 

• The amount of detail in the lower level models is significantly greater than in the upper 
level models. It became extremely difficult to keep up with everything using pencil 
and paper. 

• The strictness of the Ell DM language (i.e. its requirement to precisely define all vari- 
ables and functions, etc.) forced us to elaborate the design more carefully. 

• Most of the proofs were not very deep but had to deal with large amounts of detail. 
Without a mechanical proof checker, it would be far too easy to overlook a flaw in the 
proofs. 

• The proof support environment of Ehdm, although overly strict in some cases, provided 
much assistance in assuring us that our proof chains were complete and that we had 
not overlooked some unproven lemmas. 

• The decision procedures of ElIDM for linear arithmetic and propositional calculus were 
valuable in that they relieved us of the need to reduce many formulas to primitive 
axioms of arithmetic. Especially useful was its ability to reason about inequalities. 

Key features of the work completed during Phase 2 and improvements over the results 
of Phase 1 include the following. 

• Specification of redundancy management and the transient fault recovery scheme uses 
a verv general model of fault-tolerant computing similar to one proposed by Rushby 
[ 20 , 21 ]. 

• Specification of the asynchronous layer design uses modeling techniques based on a 
time-extended state machine approach. This method allows us to build on previous 
work that formalized clock synchronization mechanisms and their properties. 

• Formulation of the RCP specifications is based on a straightforward Maximum Fault 
Assumption that provides a clean interface to the realm of probabilistic reliability 
models. It is only necessary to determine the probability of having a majority of 
working processors and a two-thirds majority of nonfaulty clocks. 

• A four-layer tier of specifications has been completely proved to the standards of rigor 
of the EHDM mechanical proof system. All proofs can be run on a Sun SPARCstation 
in less than one hour. 

• Important constraints on lower level design and implementation constructs have been 
identified and investigated. 

Rased on the results obtained thus far, work will continue to a Phase 3 effort, which 
will concentrate on completing design formalizations and develop the techniques needed to 
produce verified implementations of R.CP architectures. 
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The following index identifies where each 
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main body of the report. Multiple entries 
appear for those names used in more than 
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processors_exist_ax 13 
reachable 22 
rec 19 

rec_maj_f_c 24 
rs_proc_state 15 
ss.update 29 
ss.update 42 
state-invariant 23 


state_rec_inv 23 
state-recovery 23 
succ 19 
succ-ax 19 
sync_thm 36 
sync_time 34 
vote_maj 20 
voted _final_state 16 
working-majority 24 
working_proc 17 
working-set 17 
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Appendix A 

LaTeX-printed Specification Listings 

The following specifications were formatted witli the assistance of the E1TDM latex-printer. 


US: Module 
Using generic.FT 
Exporting all 
Theory 

a,t: Var Pst ate 
u: Var inputs 

AfuM- Definition function [Pstate, Pstate, inputs — * bool] = 

: t = /c(u,j)) 

initial. us: function [Pstate — ♦ bool] = ( A a : a — initial, proc_state) 

End 


generic.FT: Module 

Using rcp.defs, sets [processors], cardinality [processors] 

Exporting all with rcp.defs, sets [processors], cardinality [processors] 

Theory 

us, pa, X, Y: Var Pstate 
Var processors 
it, /, q: Var nat 
u: Var inputs 
w: Var MBvec 
h: Var MBmatrix 
A , D : Var set [processors] 

maj.condition : function [set[processors] — ♦ bool] = 

( A A : 2 * card(y4) > card (fuilset [processors])) 

(* The following definitions and axioms are used to model a general class 
of f ault-tolerant computation schemes. The elaboration of these 
uninterpreted functions, as well as those in rcp.defs, would be made 
for a particular choice of application-dependent computation style and 
voting pattern. Given suitable choices, the axioms can then be shown 
to be theorems. *) 


control.state: Type 
cell: Type 
cell-state: Type 
c, d,e: Var cell 
K: Var control-state 
//: Var nat 


succ: function [control-state —*• control.state] 
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/*: function [Pstate — ► controLstate] 
ft : function [Pstate, cell — ♦ celLstate] 

/ c : function[inputs, Pstate — ► Pstate] 

/ 0 : function[Pstate — ♦ outputs] (* actuator output *) 

/,: function [Pstate — ► MB] 

f v : function [Pstate, MBvec — * Pstate] 

(• rec(c,K,H) * T iff cell c’s state should have been recovered when in 
control state K with healthy count H; note that H-l healthy frames 
will have occurred previously. *) 

rec: fu nction [cell, controLstate, n at — ► bool] 

(* dep(c,d,K) * T iff cell c’s value in the next state depends on cell d’s 
value in the current state, when in control state K; if cell c is voted 
during K, or its computation takes only sensor inputs, there is no 
dependency; if c is not computed during K, c depends only on itself; 
otherwise, c depends on one or more cells for its new value. *) 

dep: function [cell, cell, con trol.st ate — ► bool] 

dep.agree: function [cell, controLstate, Pstate, Pstate — *■ bool] = 

( A c, A', A', V':(Vd: dep(c, d t K) 3 ft(X> d) = f t (Y } d))) 


(* Axioms to be satisfied by the generic application *) 
succ.ax: Axiom A(/c(u, pa)) = succ(/*(ps)) 
full. recovery: Axiom FI > recovery-period 3 rec(c, A',//) 
initial .recovery: Axiom rec(c, K t II) D H >2 

dep.recovery: Axiom rec(c, succ(A'), H + 1) Adep(c,d, K) 3 rec(d, K>H) 
components.equal: Axiom fk(X) = fk(Y) A (Vc : / ( (A r ,c) = f t (Y, c)) 3 1 = 7 
control. recovered: Axiom 

maj .condition (/l) A (Vp : p £ A 3 w(p) = f 9 (ps)) 3 fk(fv(Y t «>)) = /*(p*) 

cell-recovered: Axiom 
maj.conditiou (A) 

A (Vp : p € A 3 u»(p) = /,(/ c (u,ps))) 

A fk(X) — K A fk(ps) = I< A dep.agree(c, K } X,ps) 

3 ft{fv(f c (u<X) } w) t c) = ft(fc(u,ps),c) 

vote.maj: Axiom maj.condition(A) A(Vp:p€A3 w(p) = fs(ps)) 

3 fv(ps, w) =ps 

(* Lemmas pertaining to sets and cardinalities *) 

card.fullset: Lemma card (fullset [processors]) > 0 
proc.extensionality: Lemma ( V p : p 6 A = p € B) 3 (A = B) 

Proof 

disharge.fi nite: Prove 

finite [processors] {/ «— ( A p — ► nat : p), N *— nrep) 
nat.nit: Sublemma k > 0 k ^ 0 
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p-nat_nit: Prove nat.nit 

p.card-fullset: Prove cardJullset from 
empty {a «— fuilset [processors]}, 
card-empty {a fuilset [processors]}, 
nat-nit { k «- card(fullset [processors])} 

p-proc-extensionality: Prove proc-ex tension ality {p «— i@pl} from 
extension ality {a <— j4, b <— B] 

End 

IIS: Module 
Using generic-FT 
Exporting all with generic-FT 
Theory 

rs-proc-state: Type = Record healthy : nat, 

procjstate : Pstate 
end record 

RSstate: Type = array [processors] of rs-proc-state 

rso’ RSstate 

rsprocO: rs-proc-state 

s, t: Var RSstate 

tt: Var inputs 

w; Var MBvec 

h: Var MBmatrix 

p, q: Var processors 

k\ Var nat 

A : Var set [processors] 


working.proc: function [RSstate, processors bool] = 

( A s,p : s(p). healthy > recovery-period) 
working-set: function [RSstate -* set[processors]] = 

( A a : ( A p : working-proc(s, p))) 
maj-Working: function [RSstate — ► bool] = 

(At: maj-Condition(working_set(t))) 

allowable- faults: functionfRSstate, RSstate -> bool] = 

( A a t t : maj.working(t) 

A ( V p : t(p). heathy >03 t(p). healthy as 1 + a(p).healthy)) 
good-values_sent: function [RSstate, inputs, MBvec — ► bool] = 

( A J.tt'tv : (V? : 

$(q). healthy >03 tn(?) = f,(fc(u,s(q). proc-state)))) 
voted-final-state: function [RSstate, RSstate, inputs, MBmatrix, processors 

— ► bool] = 

(Xs,t,u,h,p. t(p). proc-state = f v (f c (u,s(p).piocsUtc),h(p))) 


Mr.: Definition function [RSstate, RSstate, inputs — bool] = 



( A 3, t, v : ( 3 h : 

(Vp: 

j(p). healthy > 0 

3 good_values_sent(3, tt, fc(p)) 

A voted.finaLslate(3, t } «, h,p})) 
A allowable- faults ( 3 , t)) 
initial, rs: fu notion [RSstate — ♦ bool] = 

( A 3 : ( Vp : 

s(p). healthy = recovery_pcriod 

A s(p). proc_state = initiaLproc_state)) 

Proof 

End 

RS.to.US: Module 
Using RS, US, RS_majority 
Exporting all with RS, US, RS.majority 
Theory 

T 3 t 3 , t,x t y, z: Var RSstate 
u 3 ,ps,X,Y: Var Pstate 
p, i t j: Var processors 
kyly q: Var nat 
ti: Var inputs 
w: Var MBvec 
h: Var MBmatrix 
MBmatrixO: MBmatrix 

MBcons.fn: Type is function [processors — ► MBvec] 
MBfn: Var MBcons.fn 

RSstate.prop: Type is function [RSstate — ► bool] 
rs.prop: Var RSstate.prop 


RSmap: function [RSstate — ♦ Pstate] = ( A rs : maj(rs)) 
rsjneasure: function [RSstate, nat — ► nat] == (A rSyk : k) 
reachable. in.n: function [RSstate, nat -♦ bool] = 

(At,*: if it = 0 

then initiaLrs(t) 

else ( 3 3, u : reachable_in.n(3, k — 1) A Af r »(sy t, u)) 
end if) by rs.measure 

reachable: function [RSstate — bool] = ( A t : ( 3 h : reach able. in.n(i, k))) 

frame.com mutes: Theorem reachablefs) A Afrj(3, ty u) 3 A r u#(R^map(3), RSmap(t),u) 
initial. maps: Theorem initial. rs(s) 3 initial_us(RSniap(s)) 

End 

RS.majority: Module 
Using US, RS, nat_inductions 

Exporting all 
Theory 

k : Vnr nat 
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p: Var processors 
us: Var Pstate 
rs: Var RSstate 
A : Var set [processors] 


m&j.exists: function [RSstate — ► bool] — 

( A rs : ( 3 A, us : 

m&j-condition(i4) A ( V p : p € A D rs{p). proc.state = us))) 
maj: function[RSstate — ♦ Pstate] 

maj.ax: Axiom ( 3 A : 

maj.condition(A) A ( Vp : p € A D rs(p).proc.state = us)) 

3 maj(rs) = us 

End 

RSJemmas: Module 
Using RS-to.us 
Exporting all with RS.to.US 
Theory 

rs, s, t y x, y> z: Var RSstate 
us: Var Pstate 
pyiyj: Var processors 
ky ly q: Var nat 
u: Var inputs 
tu: Var MBvec 
h: Var MBniatrix 
MBmatrixO: MBmatrix 

MBcons_fn: Type is function [processors — ► MBvec] 

MBfn : Var MBcoiiB.fn 

RSstate.prop: Type is function [RSstate — *• bool] 

rs.prop: Var RSstate.prop 

m, Var proc.plus 

prop: Var function [proc.plus — ► bool] 

Cydye: Var cell 
K: Var control-slate 
H : Var nat 
A: Var set [processors] 

initiaLmaj: Lemma 

initial. rs(a) 3 (Vp : maj-exists (s) A s(p). proc.state = maj(s)) 

initial. working: Lemma initial_rs(*) 3 working_set(j) = fullset [processors] 

mitiai.maj.cond: Lemma initial. rs(s) 3 maj_condition(working.set(s)) 

control, recovery: function [RSstate — ► bool] = 

(Aa: (Vp: a(p). healthy > 1 3 fk(s(p). procjstate) = /*(maj(a)))) 
celLrecovery: function [RSstate — ► bool] = 

(Aa:(Vp,c: 

rec(c, fk{s(p) ,proc_state ) , s(p).healthy) 

3 / £ (s(p).proc_state,c) = /<(maj(s), c))) 
state.recovery: function [RSstate — ► bool] = 

(As: maj.exists(s) A control-recovery (s) A celLrecovcry(s)) 
working-majority: function [RSstate — ► bool] = 

( A s : (Vp : p € working-set (s) 3 s(p). proc-state = maj(s))) 
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consensus-prop: Lemma state-recovery (s) 3 working-majority (s) 

working, set-hefti thy : Lemma working-set(a)(p) 3 a(p). healthy > 0 

maj-sent: Lemma state-recovery(s) A good_values-sent(s, u, w) 

3 (Vp : p e working-set (s) 3 w(p) = / a (/ c (n, maj(s)))) 

rec_maj-exists: Lemma 

maj-working(s) A state-recovery ( 5 ) A A/ ra (s, t , u) 3 maj.cxists(t) 
rec-maj-f-c; Lemma 

maj-working(a) A state_recovery(s) A A f r *(s y t } u) 3 maj(l) — / c (u, inaj(s)) 

End 

RSJnvariants: Module 
Using RSJemmas, nat-inductions 
Exporting all with RSJemmas 
Theory 

rs, s, t, r, y y z : Var RSstate 

its: Var Pstate 

p, i, j: Var processors 

k, /, q : Var nat 

u: Var inputs 

u>: Var MBvec 

h: Var MBmatrix 

RSstate-prop: Type is function[RSstate — >■ bool] 

rs.prop: Var RSstate.prop 

m f n t a,b: Var proc-plus 

prop: Var function [proc-plus — ► bool] 

c t d t e : Var cell 

K: Var control-state 

//: Var nat 

A: Var set [processors] 

state-invariant: fu notion [RSstate.prop — ► bool] = 

( A rs.prop : ( V t : reachable(t) 3 rs_prop(t))) 

state-induction: Lemma 

(Vi : iiiitiaLrs(x) 3 rs_prop(x)) 

A ( V 3, t, u : reachable (s) A rs.prop(s) A i, «) 3 rs-prop(t)) 

3 state-invariant (rs.prop) 

maj_working-inv: Lemma state .invariant (maj_working) 
state-rec_inv: Lemma state-invariant (state-recovery) 

Proof 

state-invariant. to_n: function [RSstate.prop, nat — ♦ bool] = 

( A rs.prop, k : ( V t : reach able.in_n(f, k) 3 rs.prop(t))) 

base-stateJnd: Lemma 

(initial.rs(x) 3 rs.prop(x)) 3 (reachable_in.n(x, 0) 3 rs_prop(x)) 
ind-state-ind: Lemma 

(Vs,t,u : reach able (s) A rs.prop(s) A Afr*(s t <, ti) 3 rs_prop(<)) 

3 (V k : state-invariant_to-n(rs_prop, k) 

3 state_invariant_to_n(rs_prop, k + 1 )) 
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p_base_stateJnd: Prove base_stateJnd from 
reach able. in_n {t «— x y k <— 0} 

p.ind.stateJnd: Prove 

iad.stateJnd {s ♦— 5<Sp3, t «— t® p2, u ti@p3} from 

statejnvariant.to.n i «— s®p3} t 

state Jnvariant_to.n {fc «— A? 4* 1, 

reachable.in.n {t *— t, fc «— I; + 1}, 

reachable {£ *- a@p3, k «— fc} 

p.stateJnduction: Prove 
stateJnduction 
{ x «- t® p3, 
s s@p4, 
t *— t@p4, 
u u@p4} from 
naiJnduction 

{p ( \ k : statejnvariant.to.n (rs.prop, k )), 
nj «— 

base-state Jnd {x *— t@p3}, 
statejnvariant.to.n (t ♦-!, * *— 0}, 
ind_stateJnd {k +— m®pl}, 
statejnvariant.to.n {t *— t® p6, k «— k@p7], 
stateJnvariant, 
reachable {t <— t®p6} 

niaj_workingJnv.il: Lemma initiaLrs(s) D maj.working(s) 

Tnaj.workiiigJiiv.l2: Lemma ACj(si t, u) 3 maj.working(t) 

p_maj_workingJnv.il: Prove maj_workingJnv_ll from 
maj.working {£ +— a), initial.niaj.cond 

p_maj_workingJnvJ2: Prove maj_workingJnv_12 from Af r « , allowable. faults 

p.maj.workingJnv: Prove maj.working.inv from 
stateJnduction {rs.prop ♦— maj.working}, 
maj.workingJnvJl {s *— i@pl}, 

niaj_working_inv J2 {s * — sQpl, t «— /^pl, u «@pl) 
state_recJnv.il: Lemma initial. rs(s) 3 state.recovery(s) 
state.recJnv.12: Lemma 

maj.working(s) A state.recovery(s) A A f r s(s, u) A maj(t) = / c («, maj(s)) 

3 control. recovery (i) 

state_recJnv_l3: Lemma 

maj.working(s) A state.recovery(s) 

A maj(t) = / e (tt, maj(s)) 

A t(p). healthy = 1 + s(p). healthy 
A //fe(*(p).proc.state) = /fc(maj(s)) 

A /*(t(p).proc_state) — /k(maj(t)) 

A good_values_sent(s, u, h(p)) 

A rec(c,/jk(t(p).proc_state), t(p). healthy) 

3 /*(/4/c(u,a(p).procjtate) t A(p)),c) = A(/c(«, maj(s)), c) 

state.recJnv.14: Lemma 

maj.working(s) A state.recovery(s) 

A Afr S (s y t,u) A maj(i) = / c (u, maj(s)) A control. recovery (<) 

3 ccll.rccovery(t) 

state.recJnv.15: Lemma 
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reachable (s) A state. recovery (s) A M*(*« *, «) D state-recovery (t) 

p_state.recJnv_ll: Prove state_recJnv.ll from 
control, recovery, 
cell-recovery, 
state.recovery, 
initiaLtnaj {p *— p®pl}, 
initial- maj {p *— p@p2} 

p.state.recJnv-12: Prove state_rec_inv_12 from 
control-recovery {a «- <}> 

Ms {p P@ P ] }» 
control-recovered 

[ps /c(«,maj(s)), 

A «— working_sct(s), 
w ((h@p2)p@pl), 

Y — / c (ti, (s(p@pl)).proc_state)}, 
maj .sent {p ♦— p® p3, w +— ((h@p2)p®pl)}, 
maj -working {f «— a}, 
state-recovery, 

control. recovery {p ^ — p<0lpl } , 

voted .final-state {k *— h@p2, p ♦— p®pl}, 

allowable-faults {p «— p@pl) 

p_state-rec.inv_13: Prove state_rec_inv-13 from 

dep_agree {/\ - /*(maj(s)), X «- s(p).proc.state, 7 - maj(s)}, 
cell- recovered 
{ps 4 - maj(s), 
w <— A(p), 

X «— j(p).proc_state, 

A «— working.set(s), 

A' «- /fc(niaj($))}, 
maj-sent {p *— p® p2, w ^ (p) } » 
inaj.working {t <— a}, 
state-recovery, 

cell-recovery {p *— p, c ♦— d®pl], 

dep.recovery {d <— d@pl, K «— /*(maj(s)), // «— s(p).healtliy }, 
succ-ax {ps «— maj(s)} 

p.state.rec JnvJ4 : Prove stato.rec.inv.H from 
cell-recovery {s *}, 

Ms {p «- p@pi}i 

state-recJnv.13 {p *— p@pl, h *— h®p2, c +— c®p1}, 
state-recovery, 

control-recovery {p «— p@pl), 
control-recovery {s ♦- t, p «- p©pl } , 
voted.final-state {/i /i@p2, p p®|>l), 

allowable-faults {p «— p@pl), 
initial- recovery 
{ c *— c@pl, 

H — (t(p@pl)). healthy, 

A' «- /*((t(p@pl)).proc_state)}, 
succ.ax {ps <— maj(s)} 

p-state-recJnv.15: Prove state_recJnv-15 from 
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state_rcc.inv_12, 

rec_maj-exists, 

rec.maj.f_c, 

3tate.recJnvJ4, 
state-recovery {s <}, 
maj.working.inv, 

state-invariant {rs.prop «— maj.working, t «— 3 } 

p.slate-Tec-inv: Prove state.rec.inv from 
state -induction {rs-prop «— state.recovery 
sUte.rec_inv_ll {s <— r@pl), 

state.rec-inv.15 {s <— s@ pi, t «— t@ pl» u *— } 

End 

RS-top_proof: Module 
Using RSJnvariants 
Exporting all 
Theory 

ra,s y i,T>y,z: Var RSstate 
us: Var Pstate 

р, i,j: Var processors 
k t /, g: Var nat 

u: Var inputs 
u>: Var MBvec 
h: Var MBmatrix 

с, d, e: Var cell 

K\ Var controLstate 

H : Var nat 

A: Var set [processors] 

MBmatrixO: MBmatrix 

MBcons.fn: Type is function [processors -+ MBvec] 

MBfn: Var MBcons.fn 

RSstate.prop: Type is function [RSstate -* bool] 

rs.prop: Var RSstate.prop 

m,n,o,6: Var proc.plus 

prop: Var function [proc.plus — ♦ bool] 

Proof 

p_frame.com mutes: Prove frame.com mutes from 
Afus {s «— maj(s), t ♦- maj(J)}, 

rec.maj.f_c, 
consensus, prop, 
maj.working.inv, 

staleJnv&riant {rs.prop maj.working, t +- s}, 
state.recJnv, 

state invariant {rs.prop ♦— state.recovery, t <— 3 } } 
state.recovery, 

RSmap {rs s}, 

RSinap {rs «— 

p.initial.map8: Prove initial-maps from 

maj_ax {A «- working-set(j), rs «- s, us <- initiaLproc.staU-}, 
initial. us {s — RSmap(s)}, 
initial. rs {p *— ;)€Jpl }, 

RSmap {rs *— s), 
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initials maj.cond 

p_initiaLworking: Prove initial. working from 

extension ality { a «— wor king-set (s), b *- fullset [processors]}, 
initial. rs { p «— z@pl), 
working.set {p «— rQpl}, 
working.proc {p zQpl} 

p.initiaL maj.cond: Prove initial.maj.cond from 

maj .condition { A *- working.set (s)}, initial. working, card.fullset 

p.initial.maj: Prove initial_maj from 
maj.ax 
{rs «- s, 

A <— fullset [processors], 
ns «— initial.proc.state}, 
maj.exists 
{ rs ♦- s, 

A <— fullset [processors], 
us *— initial_proc_state}, 
maj.condition {A fullset [processors]}, 

initial.rs { p «— p@pl}, 
initial. rs [p «— p@p2}, 
initial.rs, 
card.fullset 

p_working.set.il eal thy: Prove working.set.healthy from 
working-set, working.proc, recovery. period.ax 

p.consensus.prop: Prove consensus.prop from 
working.majority, 

components.equal {A' «- (s(p@pl)).proc.state, Y <-maj(s)}, 
controLrecovery {p p@pl}, 
cell.recovery {p p@pl, c +- c®p2}, 
full.recovery 
{c <— c@p2, 

K <- /fc((5(p@pl))-proc3tate), 

H <- (s(p%l)).healthy), 
state-recovery, 
working-set [p *— p®pl}, 
working.proc {p p@pl}, 
recovery .period _ ax 

p.inaj-sent: Prove maj -sent from 
good.values.sent {</ «— p}, 
consensus.prop, 
working.majority, 
working.set.healthy 

p.rec_maj-exists: Prove rec.m a j .exists from 
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/ c (u,maj(s))} t 


maj.exists {ra <— t, A «— working.spt(a), us •— 

Mr, {p p®pl}. 
vote-maj 

{ps — / c (tt,maj(j)), 
w *— ((h€lp2)p@pl), 

A *— working_sct(5)} > 

inaj_sent {p +— p@ p3, tv ((/i®p2)p@pl)}, 
state-recovery, 
consensus-prop, 
working. majority {p «— p@pl}, 
votedJinaLstate {/i <— /*@p2, p +- p©pl } , 
working.set -heal thy {p «— p@pl}, 
maj-working {t s] 

p-rec.maj Xc; Prove rec-maj_f_c fVom 

maj.ax {rs *— t t A *— working.set(j), us <— / c (u f maj(s))}, 

K* {p «- p@pi}> 
vote-maj 

{p* — /c(«, maj(s)), 
w ((h@p2)p@pl), 

A «— working-set(s)}, 

maj_sent (p p©p3, w ♦— ((/i@p2)p@pl )}, 

state-recovery, 

consensus-prop, 

working. majority {p «— p@pl}, 

voted-finaLslate {h «— 4@p2, p ♦— p@pl} f 

working_set-healthy {p «— p@pl}, 

maj-working {* <— a) 

End 

RS-tcc-proof; Module 
Using rcp-defs-tcc 
Exporting all 
Theory 
Proof 

proc-plus_TCCl-PROOF: Prove proc_plus-TCCl {p *— 0} 

processors-TCCl -PROOF: Prove proccssors.TCCl {p *— nrcp) from 
processors-exist_ax 

End 

RS-to-US-tcc: Module 
Using RS_to-US 
Exporting all with RS-to.US 
Theory 

s: Var RS.RSstate 
t: Var RS.RSstate 
k: Var naturalnumber 


reachable-in_n-TCCl: Formula (-»(* = 0)) D (k - 1 > 0) 
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reachable_in_n_TCC2: Formula 

(-*(](; rs 0)) 3 rs.measure(t, k) > rs_mcasure(s, k — 1) 

Proof 

reachable.in_n.TCCl .PROOF : Prove readiable.in.n-.TCCl 
reachable.in_n.TCC2_PROOF : Prove reach able. in.n.TCC 2 
End RS_to.US.tcc 
DS: Module 
Using generic. FT 
Exporting all with gencric.FT 
Theory 

ds_proc_state: Type = Record healthy : nat, 

proc_state : Pstate, 
mailbox : MBvec 
end record 

ds.proc.array: Type = array [processors] of ds.proc-slate 
DSstate: Type = Record phase : phases, 

proc : ds.proc.array 
end record 

dso : DSstate 

dsprocO: ds_proc_state 

a, f, x, y, z: Var DSstate 

u: Var inputs 

w: Var MBvec 

i, j,p, q t qq: Var processors 

jfc: Var nat 

ph: Var phases 

A : Var set [processors] 


working.proc: function [DSstate, processors — *■ bool] = 

( A j,p : s. proc (p). healthy > recovery .period) 
working-set: function [DSstate — set [processors]] = 

( A s : ( A p : working.proc(s, p))) 
maj.working: function [DSstate — ► bool] = 

( A t : inaj.condition(working-set(f))) 

allowable-faults: function[DSstate, DSstate — ► bool] — 

( A s, i : maj.working (t) 

A ( V i : t.proc(i).healthy > 0 

3 t.proc(x). healthy = 1 + s.proc(t). healthy)) 
broadcast. received : function[DSstate, DSstate, processors — ► bool] = 
( As,t,p : {V qq : 

s.proc(</</). healthy > 0 

3 t. proc(p). mailbox (qq) = s. proc(qq). mailbox (qq))) 


A f% 0 : function [DSstate, DSstate, inputs, processors — ► bool] = 

(A s, 

s.proc(i).hea!thy > 0 

3 f.proc(i).proc.state = / r (u, s. proc (*). proc _st ate) 

A t. proc(i). mailbox (i) = /.(/c(a, s.proc(i).proc_statc))) 
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A/J,: function [DSstate, DSstate, processors bool] = 
( A s, t t i : s.proc(t). healthy > 0 

3 i.proc(*).proc_state = s.proc(i).proc_state 
A broadcast. received (s, <, *)) 


Aft: function [DSstate, DSstate, processors — bool] = 

( A 5, t, i : s.proc(s). healthy > 0 

3 (i.proc(t). mailbox = s.proc(t). mailbox 
A t.proc(i) proc_state 

= /^(s.proc^J.proc-state, s.proc(i). mailbox))) 


Aft: function [DSstate, DSstate, processors — ► bool] = 

( A s, l t « : s.proc(i). healthy > 0 

3 t.proc(i).proc_state = s.proc(i).proc^tate) 
A (t.proc(i). heal thy > 0 

3 f.proc(i). healthy = 1 + 5. proc(i). healthy)) 


Aft: function [DSstate, DSstate, inputs — ► bool] — 

( A s, t, u : maj.working(f) 

A t . phase — next_phase(s. phase) 

A ( V » : 

If s . phase — sync 
then Aft(s,i»t) 

else t.proc(i). healthy = s.proc(t). healthy 
A (a. phase = compute 3 Aft (s, u, i)) 
A (s. phase = broadcast 3 A ft(3, t, *)) 
A (s. phase = vote 3 A ft (s, t , *)) 
end if)) 


frame.N-ds: function [DSstate, DSstate, inputs — ► bool] — 

( A s, f, « : (3 z,y % z : 

Aft (a, x, u) A Aft (x, y, u) A Aft(y, x, u) A Aft (x, «))) 


initial-ds: function [DSstate — * bool] = 

(As: s. phase = compute 

A (Vi: s. proc(»). healthy = recovery .period 

A s.proc(*).proc.state = initial.proc.state)) 

End 

DS.to.RS: Module 
Using ds, rs 

Exporting all with DS,RS 
Theory 

ds,s,J,x,y, x: Var DSstate 
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rs: Var RSstate 
t, j: Var processors 
p : Var nat 
u: Var inputs 
w: Var MBvec 
h : Var MBmatrix 
MBmatrixO: MBmatrix 

MBcons.fn: Type is function [processors -+ MBvec] 

MBfn: Var MBcons.fn 

ssu.measure: function[DSstate, nat — ► nat] == (A ds,p : p ) 
ss.update: Recursive function [DSstate, nat — *• RSstate] = 

( A ds f p : if (p = 0) V (p > nrep) 
then rso 

else ss.update(ds , p — 1) 
with [(p) := rsprocO 

with [(healthy) := ds.proc(p). healthy, 

(procjstate) := ds.proc(p).proc_state]] 
end if) by ssu.measure 


DSmap: function [DSstate RSstate] = (Ads : ss_update(ds, nrep)) 

MBmc.measure: function [MBcons.fn, nat -► nat] == (A MBfn,p : p) 
MBmatrix.cons: Recursive function [MBcons.fn, nat -+ MBmatrix] = 
( A MBJn } p : if (p = 0) V (p > nrep) 
then MBmatrixO 

else M Bm at rix.cons( A/ Bfn,p — 1) 
with [(p) := MBfn(p)] 
end if) by MBmc_mcasure 


frame.commutes: Theorem 

a. phase = compute A frame-N-ds(s, t, u) 3 Ar*(DSmap(s), DSmap(t), u) 

initial-maps: Theorem initiaLds(s) 3 initiaLrs(DSmap(s)) 

good_values-sent: functionfDSstate, inputs, MBvec — *■ bool] = 

( A s t u, w : (V j : 

s. proc(j). healthy >03 tn(j) = /*(/c(u, s.proc(j).proc-state)))) 


voted-final-state: function [DSstate, DSstate, inputs, MBmatrix, processors 

— ► bool] = 

( A s, t f u, h , i . 

t . proc( i) . proc_s tate = /«»(/*(“» s. proc(i). proc-state), h(i))) 


is.new-proc-state: functionfDSstate, DSstate, inputs -> bool] = 

( A s, t, ii : ( 3 h : 

(Vi : s.proc(i).healthy > 0 

3 good-values_sent(s, u, ft(i)) 

A voted -finaLstate(s, t , u, h t i)))) 

fr-com-l: Lemma s. phase = compute A frame.N_ds(s, t, u) 

3 is-new_proc_state(s, t } u) A allowable. faults(s, t) 

fr-Com_2: Lemma is-new_proc_state(s, t , u) A a!lowable_faults(s, <) 

3 Afr» (DSmap(s), DSmap(f), u) 

fc-A: Lemma s . phase = compute A frame.N_ds(s, t, u) 

3 is.new_proc_state(s, /, u) 

fc-B: Lemma s. phase = compute A frame.N_ds(s, f, u) 3 allowable. faults(s,t) 
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End 

DSJemmas: Module 
Using DS-to.RS 
Exporting all with DS.to.RS 
Theory 

d$: Var DSstate 
rs: Var RSstate 
p, q : Var nat 
ph: Var phases 
s,t,z,y,z: Var DSstate 
Var processors 
«: Var inputs 
tv: Var MBvec 
h : Var MBmatrix 
MBfn: Var MBcons.fn 
MB. Var MBvec 

Var proc.plus 

prop : Var function [proc.plus -+ bool] 


half.frame.N_ds: function [DSstate, DSstate, inputs — bool] = 

( A x, t,u : (3 y, z : .A/d#(z,y, ti) A A/d*(y, z, ti) A A/d,.(z, 

quarter.frame.N_ds: function[DSstate, DSstate, inputs -+ bool] = 

( A y, *, « : ( 3 z : A^,(y, z, u) A A f<u(z, u))) 

fc.A.la: Lemma s . phase = compute A frame.N-ds^, f, ti) 

3 (3 = 

maj.working(x) 

A (Vi: 

x. phase = broadcast 

A x.pTOc(i). healthy = s.proc(i). healthy A z, «, •)) 

A A/dj(z f y, «) A Ad,(y, z, u) A A/dj(z,^u)) 

fc.A.lb: Lemma s. phase = compute A frame.N_ds($, t, «) 

3 ( 3 x, y, z : 

maj.working(x) 

A maj.working(y) 

A ( V * : 

x. phase = broadcast 

A x.proc(i). healthy = s.proc(i). healthy 

A^ifi.r,* M) 

A y. phase — next_phase(x. phase) 

A y.proc(i).healthy = x.proc(i). healthy 

A y,i)) 

AA/d,(»,*,») AA/d ,(*,<,«)) 

fc.A.lc: Lemma s. phase = compute A frame_N.ds(s,i,u) 
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3( 


3 z,y,z : 
inaj_working(x) 

A maj .working (y) 

A ( V i : 

x. phase = broadcast 

A x.proc(i).healthy = s.proc(i). healthy 
A 5. proc(i). healthy > 0 

3 x.proc(»)«proc_state 

= / c («, s.proc(i).proc_state)) 

A y. phase = vote 

A y.proc(i). healthy = x.proc(i). healthy 
A (x.proc(i). healthy > 0 

3 j/.proc(i).proc_state 

= x.proc(i).procjslate 
A(V>: 

x.proc(j). healthy > 0 
3 3 /.proc(*),mailbox(y) 

= / 4 (x.proc(».procjstate))))) 

A A/d,(y, z, u) A .Sfd${z,ty t*)) 


fc_A_ld: Lemma j. phase — compute A frame.N_ds(s f t, u) 

D (3 x,y, z : 

maj.working(x) 

A maj-working(y) 

A ( V jj : 

x.proc(jj). heal thy as a. proc(jji). healthy 
A (s.proc^.;). healthy > 0 

3 y . proc ( jj ) . proc_state = x.proc(.?,/).proc-state)) 

A ( V * : 

y. phase = vote 

A y.proc(t). healthy = s.proc(i). healthy 
A s. proc(i). healthy > 0 

3 y.proc(i).proc_state 

= / c (ti, s.proc(*).proc.state) 

A (Vj : 

x.proc(j). healthy > 0 
3 y.proc(i). mailbox^) 

= / a (x.proc(,y).proc_state))))) 

A A/d,(», z,u) A A fd.(z,t,v)) 


fc.A-le: Lemma s . phase = compute A frame_Njds(a, t, ti) 

D (3 x,y, z : 

maj^worklug(x) 

A maj_working(y) 

A (Vi: 

y. phase = vote 

A y.proc(i). healthy = s.proc(t). healthy 
A s.proc(i). healthy > 0 

3 y.proc(i).proc_state 

= / C (tt,s.proc(i).proc_statc) 

A (V> : 

s. pro c(j). heal thy > 0 
3 y.proc(i).mailbox(j) 

= f.{y proc(j).proc-state))))) 

A Md.(y, Z, tt) A Af d . (z, t, u)) 
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fc_A-lf: Lemma 5 . phase = cornpute A frame-N_ds(s, t, «) 

3 (3y,z : 

maj-woTking(y) 

A (Vi: 

y. phase = vote 

A y.proc(i). healthy = s.proc(i). healthy 
A s.proc(i). healthy > 0 

3 y.proc(i).proc«state 

= f c { a, 5.proe(i).proc_state) 

A(V>: 

s. proc (y) .heal thy > 0 
3 y.proc(i). mailbox (;) 

= /,(yproc(j).proc_statc))))) 

A A/dj (y, u) A A t , **)) 

fc_A_2a: Lemma s. phase — compute A frame_N-ds(s, i, u) 

D(3y t z: 

maj-working(y) 

A maj_working(z) 

A (Vi: 

y. phase = vote 

A y.proc(i). healthy = s.proc(t). healthy 
A s.proc(i). healthy > 0 

3 y.proc(t).proc_state 

= /c(ti, s.proc(i).proc-state) 

A (Vy: 

s.proc(y).healthy > 0 
3 y. proc (i). mailbox (y) 

= /<(y*proc(j).proc_state)})} 
A z. phase = nexl-phase(y.phase) 

A z.proc(i).healthy = y.proc(i).healthy 

fc_A-2b: Lemma s. phase = compute A frame_N.ds(s, u) 

3(3y,z: 

maj_working(y) 

A maj-working(z) 

A (Vi: 

z. phase = next.phase(vote) 

A z.proc(i). healthy = s.proc(i). heal thy 
A s.proc(i). healthy > 0 

3 y. proc (i).proc_st ate 

= / c (u } s.proc(i).procjstate) 

A ( V y : 

s.proc(y). heal thy > 0 
3 y. proc (i), mail box (y) 

= /•(yproc(y).proc-state)))) 
A a.proc(i). healthy > 0 

3 {z. proc (i). mailbox = y. proc (i). mailbox 
A z.proc(i).proc_stale 

— /v(yproc(i).proc_state, 
y.proc(i).mailbox)))) 

A A/j*(z, t, u)) 
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fc.A_2c: Lemma a. phase = compute A frame_N-ds(5, t, u) 

3 ( 3 y, z : 

maj.working(y) 

A majjworking(z) 

A ( V » : 

z. phase = next.phase(vote) 

A z. proc(t). healthy = j.proc(i). healthy 
A s. proc(t). healthy > 0 

3 y.proc(i).proc_state 

= f c (n t s.proc(i).proc_state) 

A z.proc(i). proems late 

= / v (y.proc(i).proc_slate, 
z.proc(»). mailbox) 

A(Vj: 

s.proc^). heal thy > 0 
3 z. proc( i). mail box (j) 

= /*(y.proc(i).proc-state)))) 

A A/d,(z,<,ti)) 

fc_A-2d: Lemma s . phase = compute A frame_N_ds(s, /, «) 

3(3 z : maj.working(z) 

A (Vt: 

z. phase = sync 

A z.proc(t). healthy = s. proc(t). healthy 
A s.proc(i). healthy > 0 

3 z.proc(i).proc_state 

= /„(/c(«, S.proc(i).proc-State), 

^.proc(i). mailbox) 

A (Vy: 

s. proc(j). healthy > 0 
3 z. proc(t). mailbox (j) 

= /,(/«(*, s.proc(jf).proc_statc))))) 

A A fd*{z,t,u)) 

fc_A_3a: Lemma j.phase = compute A frame_Njds(s 1 i, tt) 

3 ( 3 z : maj-working(t) 

A maj_working(z) 

A ( V t : 

z. phase = sync 

A z.proc(i).healthy = s.proc(»). healthy 
A a.proc(i). healthy > 0 

3 z.proc(t).procjstate 

= /4/c(ii, j.proc(i).proc-state), 
z.proc(t). mailbox) 

A ( V y : 

s.proc(ji). heal thy > 0 
3 z.proc(i).mailbox(j) 

= /.(/c(u, i.proc(j).pror_slate)))) 
A /.phase = ncxt-phase(*. phase) A t, »))) 
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fc_A^3b: Lemma a. phase = compute A frame_N_ds(s f J,u) 

3 ( 3 z : maj_working(*) 

A (Vi: 

{.phase = next.phase(sync) 

A z.proc(i). healthy = s.proc(i). healthy 
A s.proc(i). healthy > 0 

3 z.proc(i).proc_state 

= / v (/ c («, s.proc(i).proc_statc), 
z.proc(i).mailbox) 

A ( V j : 

s.proc(j). healthy > 0 
3 z.proc(i). mailbox (y) 

= fs(fc(n , s.proc(y).proc-state)))) 

A (z.proc(i).healthy > 0 

3 i.proc(»).pTOC_state = z.proc(i).proc,state) 

A (f.proc(t). healthy > 0 

3 t.proc(t) healthy = 1 + z.proc(t). healthy))) 

fc-A_3c: Lemma 5. phase = compute A frame.N<ds(s, t, u) 

3 ( 3 z : maj_working(i) 

A (Vi : 

L phase = compute 

A s.proc(t). healthy > 0 

3 <.proc(i).proc_state 

= /v(/c(«, s.proc(i).proc_state), 
z . proc( t ) . mailbox ) 

A(Vj: 

s.proc(j). healthy > 0 
3 z . proc( $ ) . m ailbox (j ) 

= /-(/c(«, s.proc(;).proc_state)))) 

A (Lproc(i). healthy > 0 

3 <.proc(t). healthy = 1 + s.proc(i). healthy))) 

fc_A_3d: Lemma s . phase = compute A frame^N-.ds(s, i , u) 

3 maj.working(i) 

A ( 3 & : (Vi : 

{.phase = compute 

A (t.proc(i). healthy > 0 

3 i.proc(i). healthy = 1 + J.proc( «). heal thy) 

A s.proc(i). healthy > 0 

3 i.proc(i).proc_state 

= /t-(/c(«, s .proc(i) . proc_state) , h(i)) 

A(Vj: 

s. proc(j). healthy > 0 

D Hi)U) = /•(/<:(“. -J.procQ'J.proc-state)))))) 

map-1: Lemma (DSmap(«)(«)) healthy = s.proc(i).healtliy 

map.2: Lemma (DSinap(s)(i))proc:-8tate = a.proc(i).proc_state 

map-3: Lemma allowable.faults(a, t) D RS(.allowable.faultsDSmap(»), DSmap(l)) 

map-4: Lemma RS(.good_values-8entDSmap(s), u, w) = good-values-sent(s, a, w) 

map-5: Lemma RS(.voted-flnal-stateDSmap(s), DSmap(l), u,h,i) 

= voted-final-state(5,l, u, h, *) 

map-7: Lemma RS(.maj.workingDSmap(s)) = D.S(.inaj-work)ng.«) 
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support. 1; Lemma (Vi : j.proc(t). healthy = x.proc(i). healthy) 

A allowable- faults(r, y) 

3 allowable-faults(j,y) 

support-4: Lemma A 3 Lphase = next-phase(s. phase) 

support-5: Lemma a. phase = ph A ph / sync AA/j,(j,i,v) 

3 ( V i : s.proc(i). healthy = x.proc(i). healthy) 

support_6: Lemma a. phase = ph 

APM sync A A/d*( 3 > x > u ) A allowable- faults (x, y) 

3 allowable- faults (s, y) 

support_7: Lemma a. phase = compute A frame_N_ds(s, t, u) 

3 ( 3 x : A fd*{s } x, u) A x. phase = broadcast A half_frame-N-ds(x, t , «)) 

support-8: Lemma x. phase = broadcast A half-framc_N_d8(x, t, u) 

3 ( 3 y : A V, «) A y. phase = vote A quarterJrame-N-ds(y, t , u)) 

support_9: Lemma y . phase = vote A quarter_frame_N-ds(y, i, u) 

3 ( 3 z : A fd»(y } z,u) A x. phase = sync A A/d # (z, tt)) 

support- 10: Lemma s . phase = sync A A/d,(s, t,u) 3 allowable- faulLs(s l /) 

support- 1 1 : Lemma 

a. phase = compute A frame_N_ds(s, f, u ) 3 allowable, fa ults(s, t) 

support-12: Lemma 

j. phase = compute A frame^N-ds(a, t , ti) 

3 ( 3 z : z . phase = sync A A /d,(s, <♦«)) 

support-13: Lemma MBmatrix-Cons(M Bfn, nrep)(i) = MBfn(i) 

support- 14: Lemma tnitiaLds(s) 3 working_set(s) = fullset [processors] 

support- 15: Lemma initiaLds(a) 3 inaj-condition (working-set(s)) 

End 

DS-top-proof: Module 
Using DSJemmas 
Exporting all with DSJeinmas 
Theory 

tls: Var DSstate 

rs: Var RSstate 

p, q : Var nat 

ph: Var phases 

s,l t x,y,z: Var DSstate 

i, Var processors 

ti: Var inputs 

u»: Var MBvec 

h: Var MBmatrix 

Ik, m, n, a, b: Var proc-plus 

prop: Var function [proc.plus — ♦ bool] 

Proof 

p-frame.com mutes: Prove frame.commutes from fr.com.l, fr.com.2 
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p-initiaLmaps: Prove initial-maps from 
initi&Lds {i p@p2}, 

initial- rs {s DSmap(a)}, 

map-1 {i <— j>®p2}, 
map-2 { i <— p@p2} 

p-fr_cotn-l: Prove fr_com_l from f<:_A, f<-B 

p_fr.com_2: Prove fr_com-2 from 
A r ra {s - DSmap(s), t DSmap(/), h - Ml |>2}, 
is_new-proc-state t *— t, i *— 

map-3 {j +— s, t «— *}, 
map.4 {$ «— 3, w *— h@p2(p@pl)}, 
map-5 {s <— a, t «— t, A 4 — J»@p2, i *— p@pl}, 
map-1 {5 +— s, 1 * p@pl} 

P-fc-A: Prove fc_A fVom 
fc_A-3d [i <- «@p2» j — i®p3}, 

is_new_proc-state {/* 

DS-to_RS.goc>d_valties-senl {tn «— 1 i 6|>1 (i®p2)}, 

DS-to-RS. voted, final-state {t — «@p2, /i — h@pl] 

p_fc_B: Prove fc.B from support-11 

p_fc-A-la: Prove fc_A.la {x <- x@pl, y 4 - y@pl, z «- *%>!} from 
Crame-N-ds, 

A/a, {$ «— s®pl, t «— x@pl}, 
next-phase {p/t ♦— a. phase}, 
distinct- phases 

p_fc-A_lb: Prove fc_A.lb {x *— xd|>l, y «— jfQpl , 2 «— -rOpl } from 
fc-A-la, Aft* {s «- x@pl, t «- y®pl}, distinct-phases 

p_fc-A-lc: Prove fc-A-lc {x — x@pl, y «- y@pl, 2 «- z®pl} from 
fc-A-lb, 

fc-A-lb {» «— j}, 

distinct-phases, 

next-phase {ph +~ x. phase) , 

ML 0*-*}. 

ML {<*-*. » *- })> 

ML {a — 1 , t 4 - y}, 

broadcast-received {*«—*, t «— y, p •— », qq «— j} 

P-fc-A-ld: Prove fc-A-ld {i ♦— i@pl, y <— t/®pl, z <— z@pl} from 
fc-A-lc, fc-A.lc {i <— jj] 

p-fc_A-lc: Prove fc-A-le {x *— z@pl, y *— y@pl, z «— z@pl} from 
fc-A-ld {jj <— j}, fc.A.ld 

P-fc_A-lf: Prove fc.A-lf {y «- y©pl, z «- z@pl} from fc-A-le 

p-fc.A_2a: Prove fc.A.2a {y <- yQpl, z «- z@pl} from 
fc-A.lf, Mda {a y®pl, t — z@pl}, distinct.phascs 

p.fc.A-2b: Prove fc_A.2l> {y *— y®pJ , z «— z@pl} from 
fc-A.2a, A fL {a «— y, l «- z, i - i@C) 

p_fc.A-2c: Prove fc.A_2c {y *— y®pl , z *— z®p1} from fc_A-2b 

p.fc_A_2d: Prove fc.A.2d {z «- z«pl) from 

fc.A,2c, next-phase {ph >- vote}, distinct-phases, fc.A.2c {i — ]) 
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p_fc_A-3a: Prove fc.A-3a { z «— 2 @pl} from 

fc.A-2d, j/d* {« — 2 ® pi, t «- *@pl}, distinct-phases 

P-fc.A-3b; Prove fc.A.3b {z *- z®pl} from 
fc-A-3a, A fj s {s + 2 , i + i®C) 

P-fc.A.3c: Prove fc.A.3c {2 — z®pl} from 

fc-A_3b, next_phase { pk sync}, distinct. phases 

p.fc.A-3d : Prove fc_A_3d 

{h +- MBmatrix.conB(( A i : z®pl. proc(»). mailbox), nrop)} from 
fc-A-3c 

i) — >® C i 

I 4 — »@c, 

II «— u@c, 
t — *®c, 

9 ♦— S®c}, 

support. 13 {A/5/n <— ( A i : z®pl, proc(»). mailbox ), t «- 1 } 

End 

DS.map.proof: Module 
Using DSJcinmas, nat_inductions 
Exporting all with DSJemmas 
Theory 

ds: Var DSstate 

rs: Var RSstate 

p, qq: Var nat 

pk: Var phases 

5 , t } x } y, z: Var DSstate 

i, y. Var processors 

«: Var inputs 

u>: Var MBvec 

/t: Var MBmatrix 

jfc, m,n, a,b: Var proc.plus 

prop : Var function[proc_plus — 4 bool] 

Proof 

mil-prop: function [DSstate, processors — * function [proc.plus — ► bool]] = 

(\ds,i: (A* : 

ss.update(ds, J:)(i), healthy 

= if * < k then ds.proc(i). healthy else rs 0 (i). healthy end if)) 
mll.base: Lemma mll-prop(s, i)(0) 

nill.ind: Lemma k < nrep A mll.prop(s, i)(k) D nill.prop(s,i)(* + *) 

p.mll.base: Prove mil -base from 
mll.prop {ds «— s, t *, k 0}, 
ss.update {ds ♦— s } p «— 0} 

p.mUJnd: Prove mllJnd from 
nill.prop {ds *- s, * 4 — *, k *— A), 
mil -prop 
{ds <— s, 
i ♦— i, 

k *— if k — nrep then nrep else k + 1 end if}, 
ss.update {ds +— s, p «— k + 1} 
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p_map_l: Prove map.l from 
DSmap {ds <— s), 

processors-induction {prop 4— HiIl-prop(s,*), n *— nrep}, 

mil-prop {ds 4— s, i 4— i, k *— nrep}, 

mil-base {s «— a, f 4— f } , 

mllJnd {s *— s, i *— i, k *— m®P2} 

ml2.prop: function [DSstate, processors -♦ function [proc.pl us — ► bool]] = 
ss.update(ds, fc)(i).proc_state 

= if i<k 

then ds. proc(i).procjBtate 
else rsofO-P* 00 -* 1 *^ 
end if)) 

ml2.base: Lemma ml2-prop(s, i)(0) 

inl2-ind: Lemma k < nrep A ml2_prop(s, i)(k) 3 nil2.prop(s, i)(k + 1) 

p-ml2_base: Prove ml 2-base from 
ml2-prop {ds *— s, i *— *, * ♦— 
ss.update {ds ♦— s, p 4— 0} 

p_nil2_ind: Prove ml2Jnd from 
ml2_prop {ds «— s, t ♦— i, fc ♦- *}> 
ml2_prop 
{ds s, 

$ — i, 

jfc if it = nrep then nrep else k + 1 end if}, 
ss-update {ds *— s, p «— k + 1 ) 

p_map-2: Prove map_2 from 
DSmap {ds «— s}, 

processors-induction {prop *— ml2_prop(s, *), n *— nrep}, 

m!2-prop {ds <— s, t *— », £4— nrep}, 

m!2_base {s *— s, i » } , 

m!2Jnd {s 4- s, * 4- i, * 4- m@P2} 

p_map_3; Prove map-3 from 

RS. allowable, faults {s 4— DSmap(s), t 4— DSinap(i)}, 

DS.allowable.faults js 4- s, t 4- t, * 4- p@pl}, 

map.7 {s 4— *}, 

map-1 js 4— s, 1 4 p@pl}, 

map-1 js 4— t, 1 *— p@pl} 

p-map_4: Prove map-4 from 

RS.good-values_sent {s 4- DSmap(s), q 4- >@P2}, 

DS_to-RS.good_values_sent {j 4— g©PlS], 

map-1 {i 4— j@ p2}, 

map-2 {t — ji@p2}, 

map.l ji 4— g@Pl), 

map-2 ji 4— g@Pl } 

p_map_5: Prove map. 5 from 

RS. voted. finaLstate {s 4- DSmap(s), i 4- DSmap(<), p 4- i}, 

DS- 1 o_ RS . voted _fi n al-state , 
map-1 {i 4— i}, 
map.l {s 4 — t 4— i], 

map-2 {1 4— *}, 
map-2 (s 4 — 1 4 i) 
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p_map. 7 : Prove map . 7 from 
proc-extensionality 

{A <— RS(.working-setDSmap(.i)), 

B 4— DS (.working-sets)}, 

RS.maj-working {< ♦— DSmap(s)}, 

RS. working-set {a *— DSmap(a), p *— p®pl}» 

RS.working.proc {a «— DSmap(a), p <— p@pl}» 

DS.maj-working {/ +- a}, 

DS. working-set {a a, p 4— p@pl}, 

DS.working-proc {a a, p 4— p@pl}, 
map-1 {i «— p@pl} 

End 

DSjsupport-proof: Modulo 
Using DS Jemmas, nat-inductions 
Exporting all with DSJemmas 
Theory 

ds: Var DSstate 

ra: Var RSstate 

p, q: Var nat 

ph: Var phases 

s,/,x,y, z: Var DSstate 

i, j: Var processors 

u: Var inputs 

w: Var MBvec 

h: Var MBmatrix 

MBfn: Var MBcons.fn 

k t m, n, a, b: Var proc.plus 

prop: Var function [proc.plus — ► bool] 

Proof 

p.support_l: Prove support - 1 {i «— i@p2} from 
DS. allowable-faults (a^-i, < +— y, * *— t@p 2 }, 

DS. allowable-faults {s 4— 5, t ♦— y} 

p.support-4: Prove support -4 from Afa» 

p_support- 5 : Prove support -5 from 
member-phases {phases-var ♦— ph], 

Mds {3 4- 8, l 4- X, u 4- u % i 4 ~ i} 

p_support-6: Prove support-6 from support- 1 , support _5 [i * - *®pl} 

p_support- 7 : Prove support _7 [x 4- x®pl} from 
frame_N-ds, 

h alf.fr ame_N.ds {x «— x@pl, y 4— y@pl, z 4— z@pl} f 
support-4 {s ♦ — s, t «— x@pl, u *— u} t 
next-phase {ph 4- compute} 

p.support-8: Prove support_8 {y *— y@pl} from 
half_frame_N_ds, 

quarter_franie_N-ds {y 4— y@pl, z *— zdpl}, 
support_4 {s x, i 4- y@pl, u 4- u}, 
next-phase {ph 4— broadcast}, 
distinct .phases 
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p.support.9: Prove 8upport_9 {z *- z@pl} fVom 
quarter.fr ame_N-ds, 
snpport-4 {s 4— y> t *— 2@pl, « <— 
next-phase {ph *— vote}, 
distinct-phases 

p.support-10: Prove support-10 from 

DS. allowable-faults, A U, {» *- »%1}, Aft, <» — *®Pl) 

p-support-1 1: Prove support- li from 

support-6 {s «-«,*♦- i@p4, y «- t, ph — compute}, 
support-6 {s — i@p4, x <- y@p5, y *- t, ph — broadcast }, 
support-6 {« — y@ p5, x «- z@p6, y — t, ph — vote}, 
support-7, 

support_8 {x ♦— x@p4}, 
support-9 {y ♦— y@p5}, 
supporl-10 (s 4— z@p6}, 
distinct-phases 

p-support_12; Prove support. 12 [z ♦— z@p3} from 

support_7, support_8 {x 4— x©pl}, support_9 {y *— y@p2} 

si 13-prop: function [MBcons.fn, processors function[proc-plus — bool]] 
(A MDfn.i : (Xk : 

MBinatrix_cons(M£/n, k)(i) 

= if t < k then MBfn(«) else MBmatrixO(i) end if)) 

si 13-base: Lemma sll3-prop(A/tf/u, t)(0) 

sl!3_ind: Lemma m < nrep A sll 3_prop( A/ Z?/n, i)(m) D 
si 1 3-prop (M Bfn, t)(m + 1) 

p.sll 3-base: Prove sll 3-base from 

si 13_prop {Jfc ♦— 0, i 4- »}, MBmatrix.cons {p 4— 0} 

p_sll3Jnd: Prove sll3_ind from 
s!13_prop {Jb «— m, i 4— i}» 

sll 3-prop {t 4- i, k 4- if m = nrep then nrep else m + 1 end if}, 
MBmatrix-cons {p 4- m + 1} 

p_support-l 3: Prove support. 13 from 

processors-induction {prop <— sll3_prop(A/ Bfn } i), n ♦— nrep}, 

si 13-prop {k 4- nrep, » 4- i), 

sll 3-base {i 4— t) , 

sll3_ind {t 4 - i % m — m®pl} 

p.support.14: Prove support_14 fVom 

proc.ex tension ali ty {^4 4— working.set(s), B 4— full set [processors]}, 
initial-ds {t 4— p@pl}, 

DS. working-set {p 4— p@pl}, 

DS.working.proc {p 4— p®pl} 

p.support-15: Prove support_15 from 

maj.condition {^4— working_set(s)}, support. 14, card.fullset 

End 

DS-to.RS-tcc: Module 
Using DS-to_RS 
Exporting all with DS.to.RS 
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Theory 

ds : Var DS.DSstate 
p: Var natural number 

M Bfn: Var fu notion [rcp.defs. processors — ► rcp.defs. MBvec] 
ss.npdate.TCCl: Formula H(j> = 0) V (p > nrep))) 3 (;> - 1 > 0) 

ss.update.TCC2: Formula (->((p = 0) V (p > nrep))) 3 ((p > 0) A (p < nrep)) 

ss-updatc.TCC3: Formula 

(-1 ((j> = 0) V (p > nrep))) D ssu.measiire(ds, p) > ssu.measur c(</*,p - 1) 

MBmatrix.con8.TCCl: Formula 
H(f> = 0) V (p > nrep))) 

3 MBmc_measure(A/ Bfn y p) > MBinc_measure(Af B/n , p - 1) 

Proof 

ss_update_TCCl .PROOF: Prove ss.update.TCCl 
ss.update.TCC2-PROOF: Prove ss.update.TCC2 
ss_update.TCC3_PROOF: Prove ss.update.TCC3 
MBmatrix-cons-TCCl-PROOF: Prove MBmatrix.cons.TCCl 
End DS-to_RS-tcc 
DS-Support-prooLtcc: Module 
Using DS_supporl_proof 
Exporting all with DS_support_proof 

Theory 

p: Var rcp.defs. processors 
m: Var rcp.defs. proc.plus 
z: Var DS.DSstate 
y: Var DS.DSstate 
z: Var DS.DSstate 
i: Var rcp.defs. processors 

p-sl 1 3_ba8e_TCC 1 : Formula ((0 > 0) A (0 < nrep)) 
p_sll3_ind-TCCl: Formula 

(( if m = nrep then nrep else m + 1 end if > 0) 

A ( if m as nrep then nrep else m + 1 end If < nrep)) 

p.support-13-TCCl : Formula ((nrep > 0) A (nrep < nrep)) 

Proof 

p_sl 1 3-base_TCC 1 -PROOF : Prove p.sll3.base-TCCl 
p-sl 13-ind-TCCl -PROOF: Prove pjril3Jnd.TCCl 
p-support- 1 3-TCC 1 .PROOF : Prove p-Support.13.TCCl 
End DS-support.prooLtcc 
DS.map.proof-tcc: Module 
Using DS-map.proof 
Exporting all with DS-map.proof 
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Theory 

k: Var rcp-defs.proc-plus 
q: Var rcp.defs. processors 
j: Var rcp.defs, processors 
p: Var rcp.defs, processors 
m: Var rcp-defs.proc_plus 

p_ml 1 -base.T CC 1 : Formula ((0 > 0) A (0 < nrep)) 

p-ml'Und-TCCl: Formula 

(( if jb = nrep then nrep else k - f 1 end if > 0) 

A ( if k = nrep then nrep else k + 1 end if < nrep)) 

p.map.l-TCCl: Formula ((nrep > 0) A (nrep < n rep)) 

Proof 

p.mlJ -basc-TCCi -PROOF : Prove pjnll.ba8e.TCGl 
p_mllind-TCCl_PROOF: Prove p.ml].iud.TCCl 
p_map-l-TCCl_PROOF: Prove p_map_i-TCCl 
End DS-map-proof_tcc 

DA: Module 

Using clkmod, generic- FT 
Exporting all with clkmod, generic.FT 

Theory 

max_comm-delay : realtime (* Max broadcast delivery tine *) 
da-proc-state: Type = Record healthy : nat, 

proc-state : Pstate, 
mailbox : MBvec, 

Iclock : logical-clock time, 

cum_delta : number (* = Corr; added to logical 
end record to obtain physical *) 

da_proc-array: Type = array [processors] of da-proc_state 


DAstate: Type = Record phase : phases, 

sync-period : nat, (* = idealized frane count ♦) 
proc : da-proc-array 
end record 

s t l } x t y } z y da: Var DAstate 

«: Var inputs 

w: Var MBvec 

i, j, p, 7, qq : Var processors 

Jb: Var nat 

ph\ Var phases 

ps: Var da_proc_state 

T: Var logical, clock time 

A: Var sct[processors] 


Corr_iniplementation: 


Lemma s.proc(|i).cum.dclta = Corr 


(j. sync-period) 

p 
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working_proc: function [DAst ate, processors — > bool] = 

(A s,p : s.proc(p). healthy > recovery-period) 
working-set: function [DAstate — ► set[processors]] = 

( A s : ( A p : working_proc(s, p))) 
maj-working: function [DAstate — ► bool] = 

(At: maj-condition(working_set(t))) 

enough-hardware: function [DAstate — * bool] = 

(At: maj_working(<) A enough .clocks (f. sync-period)) 

da-rt: fu nction [DAstate, processors, logical-dock time — ► realtime] = 

(A da, p, T : c p (T + da.proc(p).cum_delta)) 
unknown: fraction 

v\ fraction = unknown (* variability of processor run rates *) 

A\ Y: Var logical.clocktime 
D : Var number 

clock-advanced: function [logical-clock time, logical-dock time, number 

— ♦ bool] = 

( A X, Y, D : X + D ♦ (1 - v) < Y A Y < X + D * (1 + *)) 
duration: functiou [phases — ► logical.clocktime] 

broadcast-duration: Axiom 

(1 - Rho) ♦ (duration (broadcast) -2 * v * duration(compute) - v ♦ duration (broadcast)] - 6 
> max.comm-delay 

broadcast_duration2: Axiom 

duration (broadcast) - 2 * v * duration (compute) — v * duration (broadcast) > 0 
all-durations: Axiom 

(1 i/) * duration (compute) + (1 + i/) * duration (broadcast) < frame-time 

pos-durations: Axiom 

0 < (1 — v) * duration (compute) 

A 0 < (1 - i/) * duration (broadcast) 

A 0 < (1 - v) * duration (vote) A 0 < (1 - v) * duration (sync) 

broadcast- received: function [DAstate, DAstate, processors — ► bool] = 

( As,f,p : (Vqq : 

s.proc(gg). healthy > 0 

A da-rt(s, qq y 5 .proc(gy).lclock) + max_comni-delay 
< da_rt(J,p, f.proc(p).lclock) 

3 t.proc(p).mailbox(gg) = s. proc(qq). mailbox (qq))) 


A r j a : function [DAstate, DAstate, inputs, processors — ► bool] = 
(As,/, u, i : 

s.proc(i). heal thy > 0 

3 /.proc(i)-proc-state = / c (ti, s.proc(i).proc_state) 

A /.proc(t). mailbox (i) = fs(fc(u, s.proc(i) proc_state))) 


A/J 0 : function [DAstate, DAstate, processors — ► bool] = 
( A s, I, i : s.proc(i). heal thy > 0 

3 /.proc(i).proc_state = s.proc(t).proc_slate 
A b road cast .received (s, /, i)) 
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A/a a : function [DAstate,DAst ate, processors -♦ bool] = 

( A s, ty i : s.proc(i).healthy > 0 

3 fproc(i), mailbox - s.proc(«). mailbox 
A t.proc(*).proc.state 

= /v( 3 .proc(t).proc_state, s.proc(i).mailbox)) 


Af d V function [DAstate, DAstate, processors — bool] = 

( A s t ty i : s.proc(i). healthy > 0 

3 t,proc(*),proc^state = s.proc(i)proc_slate) 

A ((.proc(t). healthy > 0 

3 J.proc(i). healthy = 1 + s.proc(t)*hcalthy 
A nonfaulty_clock(i, t.sync.period)) 

A i.sync.period = 1 + s.sync.period 
A (nonfaulty_clock(t, s.sync.period) 

3 i.proc(»).lclock = (1 + s. sync-period) * frame_tiine 
A i.proc(i).cum.delta 

= 4 .proc(<).cum.delU + A< a8 y nc - period) )) 


A fda- function [DAstate, DAstate, inputs — bool] = 

(A s,t } u : enough.hardware(f) 

A f. phase = next_phase(s. phase) 

A ( V » : 

if s. phase = sync 
then A/2 a (s,M) 

else f.proc(i). healthy = s.proc(s). healthy 
A t.proc(i).cum-delLa = s.proc(*).cum_delta 
A t. sync-period = a. sync-period 

A (nonfaulty .clock (i, s.sync.period) 

3 dock . ax! vanced ( s . proc ( i) .Iclock , 
i.proc(i).lclock, 
duration (s. phase))) 
A (s. phase = compute 3 t , u, i)) 

A ( 5 . phase = broadcast 3 A/J 0 (s, /»i)) 

A (s. phase = vote 3 A/J 0 (s, t f »)) 

end if)) 


initial.da: function [DAstate — ♦ bool] = 

(As: s . phase = compute 
A s. sync.period = 0 
A ( V i : 

s.proc(i). healthy = recovery, period 

A s .proc(t) . proc_st ate = initial. proc_slatc 
A s.proc(t)xum_ddta = 0 

A s.proc(i).lclock = 0 A nonfaulty .dock (t, 0))) 


End 

DA.to.DS: Module 
Using DA,DS 
Exporting all with DA, DS 
Theory 
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da, s,i,x,y, Var DAstate 

ds : Var DSstate 

p, »,jf: Var processors 

it,/: Var nat 

u: Var inputs 

tn: Var MBvec 

h: Var MBmatrix 

ph: Var phases 

MBmatrixO: MBmatrix 

MBcons_fn: Type is function [processors — ♦ MBvec] 

MBfn : Var MBcons.fn 
T, T\ , T 2 , Var logical. clocktime 
DAstate.prop: Type is function [DAstate — ► bool] 
da-prop: Var DAstate.prop 

da_measure: function [DAstate, nat — ► nat] == ( A da,k : *) 
ss.update: Recursive function [DAstate, nat DSstate] = 
(Ada,* : if (* = 0) V(* > nrep) 
then dso 

else ss.u pd ate (da , * — 1) 
with [(proc)(*) := dsprocO 

with [(healthy) := da.proc(*).heakhy, 

(proc_state) := da.proc(*).proc_statc, 
(mailbox) := da.proc(fc). mailbox]] 
end if) by da_measure 


DAmap: function [DAstate — ► DSstate] = 

( A da : ss_update(da, nrep) with [(phase) := da. phase]) 
MBmc.measure: function [MBcons.fn, nat — ♦ nat] == (A MBfn,k : *) 
MBmatrix.cons: Recursive function [MBcons.fn, nat — ► MBmatrix] = 
( A MBfn, k : if (* = 0) V (* > nrep) 
then MBmatrixO 

else MBmatrix.cons(Af £?/n, * — 1) 
with [(*) := MBfn(*)] 
end if) by MBmc.measure 


reach able.in.n: function [DAstate, nat — ► bool] = 

(At,*: if * = 0 

then initial.da(t) 

else (3s,u : reachable.in_n(s, * - 1) A A/d«(s, t, u)) 
end if) by da.measure 

reachable: function [DAstate — bool] = ( A t : (3 * : reach al»le.in.n(f, *))) 
phase.commutes: Theorem reachable(s) A A/da(«» t , «) D A/dj(DAmap(s), DAmap(t), u) 
initial. maps: Theorem initiaLda(a) 3 iniliaLds(DAmap(s)) 

End 

DA Jnvariants: Module 
Using DA.to.DS, nat.inductions, DA Jemmas 
Exporting all with DA.to.DS 
Theory 
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da } $ y t, x ,y, z: Var DAstate 

ds: Var DSstate 

p r i, j\ Var processors 

jfc, /: Var nat 

u: Var inputs 

w: Var MBvcc 

h : Var MBmatrix 

ph: Var phases 

cdv: Var number 

it: Var period 

T, Ti , T 2 , Var logical .clocktiine 
DAstate.prop: Type is function [DAstate — ► bool] 
da.prop: Var DAstate.prop 

state-invariant; fu action [DAstate_prop — * bool] = 

( A da-prop : ( V t : reachable(t) 3 da.prop(t))) 

state-induction; Lemma 

(V x : initiaLda(x) 3 da-prop(x)) 

A (V5,l,ii : reachable(s) A da_prop(s) A A Ci«(s f f, u) 3 da-prop(t)) 
3 state-invariant (da-prop) 

enough-inv: Lemma state-invariant(( A 3 : enough.hardware(s))) 

iiLclks; function [DAstate — ► bool] = 

(As: (Vi; 

s.proo(i),heaJthy >03 nonfaulty-clock(i, s.sync_period))) 

nfclk-inv: Lemma state-invariant (( A s : nf-clks(s))) 

Iclock-eq: function [DAstate bool] = 

( A j : ( V i, j : 

non fau 1 ty -clock ( i , s .sy nc.period ) 

A nonfaulty .clock (j, s.sync.period) A s. phase = compute 
3 s.proc(i).lclock = s.proc(j).Iclock)) 

Iclock-inv: Lemma stateJnvariant(( A s : lclock.eq(s))) 

Iclock-val: function [DAstate —*■ bool] = 

( A s : ( V i : 

non faulty-dock(i, s.sync.period) A s. phase = compute 
3 s.proc(t) Icfock = s.sync.period * frame.time)) 

clkval.inv: Lemma stateJnvariant(( A s : Iclock_val(s))) 

rtll: Lemma reach able (da) A non faulty _clock(p, da.sy nc.period) 

3 da.proc(p).cuni .delta = Corr^ sync - period) 
da rtJem: Lemma reach able (da) A nonfauliy-clot:k(p, d«.sync_period) 

3d a .rt(rfa >Pl T) = r < < rf “ SynC ’ ,>eri0d) m 

cum-delta_val; function [DAstate — ► bool] = 

(As: (Vp : 

nonfaulty .clock (p, s.sync.period) 

3 s.proc(p)xum_ddta = Corr^ 9 • S 5 qi<: -P cr * 0 ^l)) 

CorrJem: Lemma ii > 0 3 Corr \ = Corrp' n 

cdll : Lemma A/i < *(s t t , u) A s.proc(p).cum.delta = cdv 
3 t.proc(p).cum.delta = cdv + 

cu m _delta-in v: Lemma stab -invariant (( A s : cum_delta.val(s))) 
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Proof 

state-invariant-to_n: function [DAstate.prop, nat — ► bool] = 

( A da_prop, k : ( V t : reach able-in_n(i, &) 3 da-prop(l))) 

base-stateJnd: Lemma 

(initiaLda(x) 3 da_prop(x)) 3 (reachablc_in-ii(x t 0) 3 da_prop(x)) 
ind_state_ind: Lemma 

( V s } *, u : reachablc(s) A da-prop(s) A A/da(«, t, «) 3 da-prop(/)) 

3 ( V ifc : stateJnvariant_to_n(da_prop, k) 

3 state_invariant_to_n(da-prop, k + ])) 

p_base-state_ind: Prove basejstate_ind from 
reach ableJn.n {t <— z, k *— 0} 

P-ind-stateJnd: Prove 

ind-stateJnd {a — s@p3, t «- <@p2, u ♦- a@p3} from 
stateJnvariant_to_n {k *— fc, t ♦— s@p3}, 
staleJnvariant_to-n [k *— k + l, 
reach able-in_n {i +- (, k *— k + 1], 
reachable {J «— a@p3, k «— k) 

p.stateJnduction: Prove 
state-induction 
{x 4— i<0)p3, 
s *— a@p4, 

{ «— <@p4, 
u «— «®p4) from 
nat-induclion 

{p «— ( A k : stale-invariant- to_n(da_prop, fc)), 

w 2 <— fc@p7}, 

base_stateJnd {x «— i@p3}, 
state-invariant. to_n {l i, «— 0}, 
ind-state-ind {k m^pl}, 

3tateJnvariant_to_n {* «— <@p6, k «— fc@p7}, 

state-invariant, 

reachable {i (®p6} 

enough-inv_ll: Lemma initiaLda(s) 3 enough-hard ware(s) 

enough-inv-12: Lemma Afda(&> «) A enough-hardware(a) 3 enough-hard ware(J) 

p-enough-inv-11: Prove enough-invJl from 
enough-hardware {< «— s} t 
enough-docks {i «— s.sync-pcriod }, 

DA.maj-working {< <— s}, 
support_14, 
support-15, 
processors_exist-ax 

p_enough-inv J2; Prove enough JnvJ2 from A 

P-enough_inv: Prove enough.inv from 

stateJnduction {da_prop «- ( A s : enoughJiardware(s))}, 
enough-invJl {s «— x@pl}, 

enough_inv_12 {s «— 3@pl, t <— f@pl, u <— u@pl) 
nfdkJnvJl: Lemma initiaLda(s) 3 nf.clks(s) 
nfclk-inv-12: Lemma A/d a (s, «) A uLclksf*) 3 rif-dks(J) 
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p.nfclkjnv.ll: Prove nfclkjnv.il fVom nf.clks, initial.da (i i@pl} 

p.nfclk Jnv_l2: Prove nfclkJnv_12 fVom 
kdo. {» — «@p3}, nf.clks {» — »@p3), uf-clks (s — <}, A f‘ ta {i — i@p3} 

p.nfclkJnv: Prove nfclkJnv from 

stateJnduction {da_prop «— ( A s : nf.clks(j))}, 
nfdkJnv.il <— r@pl}, 

nfclk.inv.12 {s «- s@pl, / «- /®pl, « «- «®pl) 

Iclock.inv J l : Lemma initiaLda(s) 3 klock.cq(s) 
lclock-.inv.l2: Lemma A fda(s y /, u) A s.phase = sync 3 klock.eq(f) 
lclock_invJ2b: Lemma 

M da ($ y t , u) 3 (5. phase = sync 3 /. phase = compute) 

A (s. phase = compute 3 /- phase = broadcast) 

A (s. phase — broadcast 3 /.phase = vote) 

A (a. phase = vote 3 /.phase = sync) 

pJclock_inv_12b: Prove lclockJnvJ21> from 

I 

distinct, phases, 
next-phase {ph <— compute}, 
ncxt_phase {ph <— vote}, 
next.phase {pA «— broadcast}, 
next.phase {pfc «— sync} 

lclock_invj2c: Lemma A/d a («, /, «) A s. phase ^ sync 3 /.phase ^ compute 

p_lclock.inv.l2c: Prove lclock.inv.l2c from 
lclock.inv.12b, 
distinct .phases, 

member-phases {phases.var <— /.phase}, 
member-phases {phases.var «— s. phase} 

lclock_inv_l3: Lemma 

A/aa(5,/» u ) A s-phase / sync 3 /.sync.pcriod = s.syiic_peno<! 
lclock.inv.14: Lemma 

A/da(3, /, «) A 3 . phase ^ sync A lclock.eq(s) 3 lclock.eq(/) 

pJclock.inv.ll: Prove lclock.inv.il from 

Iclock-eq, initial.da {» «- i@pl}, initial.da {i <- >®pl} 

p_lclock_inv_12: Prove Iclock.inv J2 from 
Iclock.eq {s «— /}, 

Mda {l - I@P1}, 

A/da {* Pl}» 

A/da {* *“ *®PU. 

A/Ja {* 

nfcJem {p «- i@pl, * — s.sync.period), 
nfcJcm {p «- jOpl, i «- s.syut .period} 

p.lclockJnv_13: Prove lclock.inv_l3 from A/da {« «- *}, Iclock.inv J2b 

pJclockJnv_14: Prove lclock.inv.14 from 
lclock_eq {s /}, 

Iclock.eq {i t@pl, j <— i®pl}. 

lclock.inv.12c, 

distinct, phases, 

iclock.invJ3 



p_lclock_inv: Prove IcIockJnv from 

stale-induction {da_prop 4— ( A a : lclock-eq(a))}, 
ldodc-inv.ll {a «— x®pl}, 

lclock-inv_12 {a a® pi, t *— pi, ti 4- u@pl}, 

lclock-inv_l4 {a ♦— a©pl, t *— u 4— u®pl} 

dkval_inv.ll: Lemma initiaLda(a) 3 lclock_vaI(a) 

clkval.invJ2: Lemma reachable(a) A.A/da(*i t* «) 3 block. val(J) 

p-clkval.inv.ll: Prove clkval.inv_ll 

from klock-val, initiaLda {* <— t@pl) 

P-dkval.invJ2: Prove clkval_invJ2 from 
Iclock.val {5 <— <}, 

A/’da {• «“ 

AfL 

supporl-16 {ph 4— s. phase}, 
prev-phase {ph *— t. phase) , 
nfcJem [p <- t@pl, * 4- a.sync_period } 

p.clkvaLinv: Prove dkval-inv from 

stateJnduction {da.prop +- ( A s : lclock-val(a))}, 
clkvaLinvJl {a *— x@pl}, 

clkval-inv_12 {a 4— a@pl, t *— pi, u <— «®pl} 

p_rtll: Prove rtll from 
cu in-del ta_inv, 

state-invariant {da-prop 4- ( A 3 : cum-ddta-val(a)), t «- da}, 
cum-delta-val {a <— da} 

p.da-rtJem: Prove da_rtJem from 

da.rt {p — p], rf ( *j 2) (*3) {i <- do.sync.period, p <- p], rtll 

cum-delta-invJl: Lemma initial.da(a) 3 cum_delta_val(a) 

p_cum-delta.inv_ll: Prove cum>delta_inv.ll from 

initial_da [i 4- p®p2}, cum.delta-val, Corr ( ** 2) {p *- p®p2, i *- 0} 

cum-delta.inv.12: Lemma 

A/d«(a, t, u) A a. phase = sync A cum-ddta_val(s) 3 cum.ddta_val(t) 
pt,pa: Var period 

cdi_12a: Lemma pt = ps + 1 3 Corr ( p * >i) = Corr [ , pd) + Ap ps * 

p_cdiJ2a: Prove cdiJ2a from Corr ( ** 2) {i 4- pt, p — p] 

p_cum-delta-invJ2: Prove currudelta_inv J2 from 
cuni-delta-vai {a 4— a, p *— p®p2}, 
cum-delta_val {a 4— t}, 

A/'da {* «- P%2}, 

ML {* P@P2}, 

cdi_12a {p 4- p®p2, pt 4- Lsync_period, ps +- *.syuc_period }, 
nfcJem {p 4- p@p2, t 4- s.sync_period} 

cum-delta_invj4: Lemma 

A/da(«» ti) A a. phase / sync A cum_ddta_val(a) 3 cum -delta, val(t) 
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p_cunudeltaJnvJ4: Prove cum-deltaJnvJ4 from 
cum-ddta-val {j <— <}, 
cum -dell a_val {p <— p@pl}, 

Mda {* P@ PUi 
distinct-phases 

P-cmn-delta-inv: Prove cum_della_inv from 

stale-induction {da_prop «- ( A a : cum_delta_vai(s))}, 
cum_delta_inv_ll {s x@pl), 

cum-delta_inv .12 {s *— s@pl, t i@pl, « ♦— u®pl}» 
cum_delta_inv_14 {a «— s@pl, t *— l@pl, u *— u©pl) 

End 

DAJemmas: Module 
Using DA-to_DS f clkprop 
Exporting all with DA_to_DS,clkprop 
Theory 

ds: Var DSstate 
da: Var DAstate 
k: Var nat 
ph : Var phases 

z: Var DAstate 
ss, tt: Var DSstate 
Pt Qj «, 3' Var processors 
u: Var inputs 
u>: Var MBvec 
h: Var MBinatrix 
MBfn: Var MBcons-fn 
m,n,a,6: Var proc.plus 
prop: Var function [proc.plus — ► bool] 

T i Ti,T 2 i BB t bb: Var logical. clockti me 
DAstate.prop: Type is function [DAstate -♦ bool] 
da.prop: Var DAstate.prop 

phase.com.compute: Lemma 

s. phase = compute A A Cia(a, t , u) D A/d,(DAmap(s), DAinap(J), n) 

hidel: function [DAstate, DAstate, inputs — ► bool] = 

( A s } t, u : (enough-hard ware(f) 

A t. phase = next_phaso(s. phase) 

A J.sync.period = s. sync. period 
A ( V * : 

f. proc(t). healthy = s.proc(»). healthy 

A t.proc(i).cum_delta = s.proc(i).cum_dclta 
A sync.period = s.sync.period 

A (non faulty _clock(i, s. sync .period) 

3 clock-advanced (s.proc(t).lclock, 
t.proc(i).lclock, 
du ration (s. phase))) 

A -V£ 0 (*,l. »,«')))) 

pliase.coniJxl: Lemma a . phase = compute A Afda(a, i , «) D hidel (s, t, u) 
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phase-comJx2: Lemma 
3 . phase = compute 

A (maj_working(DAmap(t)) 

A ( V * : 

DAmap(<).phase = next_phase(DAmap(s).phase) 

A DAmap(f).proc(i). healthy = DAmap(s).proc(i). healthy 
A A/£(DAmap(j), I)Amap(t), tx, *))) 

3 A/d,(DAmap(s), DAmap(t), u) 

phase_comJx4: Lemma 
s. phase = compute 

A (maj_working(DAmap(<)) 

A ( V i : 

t. phase = next_phase(s. phase) 

A tproc(i). healthy = s. proc(i). hcaltliy A A/J a (s, t, «,*))) 

3 A/dj(DAmap(s), DAmap(/), u) 

phase_com_lx7: Lemma 

5 . phase = compute A A fda{»> u) 

3 (maj.working(DAmap(t)) 

A ( V i : 

t. phase = next-phase (s. phase) 

A f.proc(i). healthy = 3. proc(t), healthy A A/j a (s, »))) 

phase.com-broadcast: Lemma 

reachable(s) A s . phase = broadcast A A/*da(s, «) 3 A f d,(DAmap(3), DAmap(t), «) 

corn-broadcast. 1: Lemma 

s . phase = broadcast A A fda(s, <, ti) 3 ( V i : A/J a (s, t, *)) 

com .broadcast. 2: Lemma 
5. phase = broadcast 
A reachable(s) 

A s.proc(t). healthy = J.proc(»). heal thy 
A A fda(s, t* tl) A A fda(s> *) 

3 A/J,(DAmap(3), DAmap(t), i) 

com_broadcast_3: Lemma 
S3, phase = broadcast 

A It. phase = next_phase(ss. phase) 

A ( V t : Af% s (ss } tt } «) A tt.proc(i). healthy = ss.proc(s). healthy ) 

A DS.maj-working(it) 

3 A/d,(ss,tt,u) 

com_broadcast_4: Lemma 

s.ph^se = broadcast AA/da(s,<yu) 

3| DAmap(t). phase = next_phase(DAmap(s). phase) 

A DAmap(t).proc(i). healthy = DAmap(s).proc(i). healthy 
A DS.maj_working(DAmap(<)) 

coin_broadcast_5: Lemma 
reachable(s) A A/da(s, t, u) 

A s . phase = broadcast 

A s. proc(i'). healthy > 0 A broadcast_received(s, t, «) 

3 broadcast-received(DArnap(s) t DAmap(t), i) 

phase-com_vote: Lemma 

s . phase = vote A A/ja (s, t, u) 3 A/d,(DAmap(s), DAmap(l), n) 
com_votc_l : Lemma s . phase = vote A A/a«(s, i, u) 3 ( V i : A/J 0 (s, /, t)) 
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com_vote- 2 : Lemma A /J a (s, t, i) 3 A/X(DAmap(s), DAmap(i), i) 


com-vote-3: Lemma 

ss . phase = vote A tt. phase — next_phase(as. phase) 

A (Vi : A tl.proc(t). healthy = ss.proc(i). healthy) 

A DS.majjworking(ff) 

D Afd»{ss } tt } u) 

com-vote_4: Lemma 

s , phase = vote A Afda(s, t f «) 

3 DAmap(f). phase = next_phase(DAmap(s). phase*) 

A DAmap(t) proc(i). healthy = DAmap(s).proc(i).liealthy 
A DS,inaj-working(DAmap(f)) 

phase_com_sync: Lemma 

s. phase — sync A A fda(s* t, a) D A/d a (DAmap(s), DAmap(f), «) 
com-sync_l: Lemma s. phase = sync A A/do(*,<, a) D (V i : Af£ a (s> t, t)) 
com_sync_2: Lemma A/j 0 (s, t t i) 3 Ad,(DAmap(s), DAmap(i), *) 
comjsync-3: Lemma 

ss. phase — sync A if, phase = ncxt_phase(ss. phase) 

A(V» : SSj,(ss,tt, i) A DS.maj.working(ii)) 

D A/d,(ss, tt, u) 

com-sync_4: Lemma 

s. phase = sync A A/da($, t, tt) 

D DAmap(t). phase = next-pha.se(DAmap(a). phase) A DS.maj.working(DAmap(t)) 

earliest_Iater_time: Lemma 
T 2 = T, -f BB A (Ti > T°) 

A (BB > T°) 

A nonfaulty_clock(», da.sync-period) 

A nonfaulty-dock(>, da. sync-period) 

A enough-clocks(da. sync-period) 

AT 2 € ^ do S y nc -P er > ocl > a Ti € 9 ync.period) 

D r< (rfa,sync.period)^ 

> rt (<*- sy«C-P«iod)( Tj ) + (j _ Rho) * \BD\ - 6 

ELT: Lemma T 2 > Ti + bb 
A(Ti >T°) 

A (bb > T°) 

A nonfaulty_clock(p, da,sync_period) 

A non faulty _dock(<j, da.sync.period) 

A enough_clocks(da.sync-period) 

AT 2 € ^<*" 9 y nc -P erio<i ) aTi € if( da S y nc “P erio<i ) 

_ j.ia.sync-period),™ v 

Drip J (I 2 ) 

> H (-<. sync-pcriod) (7 . } + ( , _ Rho) , (fc6 | _ 6 

elt_a: Lemma (bb >T°) A BB >66 3(1 — Rho) * \BB\ > (1 — Rho) * |66| 

map.l: Lemma DAmap(s).proc(t). healthy = s.proc(t). healthy 

inap-2: Lemma DAniap(s).proc(i).proc_state ss s.proc(i).proc_slatc 

niap.3: Lemma I)Ainap(s). phase = s . phase 

map-4: Lemma DAniap(s).proc(t). mailbox = s.proc(i). mailbox 

map_7: Lemma DS( jnaj-workingl)Ainap(s)) = I)A(.inaj-workings) 
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support- 1: Lemma initial.da(s) 3 world ng-set(s) = fullset [processors] 

support-4: Lemma a. phase = ph AA(ia(s,i,ti) D x. phase = next.phase(ph) 

support_5: Lemma a. phase — ph A ph ^ sync A A Cia(s, x, u) 

3 (Vi : s. proc(i). healthy = x.proc(i). healthy) 

support_13: Lemma MBmatrix_cons(Af Bfn, nrep)(i) = MBfn(i) 

support-14: Lemma initiaLda(s) 3 maj_condilion(working-set(s)) 

support-15: Lemma initial-da(.s) 3 nu in .good .clocks (s.sync.period, nrcp) = nrep 

support-16: Lemma prev.phajse(next_phase(p/i)) = ph 

End 

DA.top.proof: Module 
Using DA Jemmas, DA-in variants 
Exporting all with DAJenimas 
Theory 

ds: Var DSstate 
da: Var DAstate 
k: Var nat 
ph: Var phases 

Var DAstate 
as, tt: Var DSstate 
p t q t i t j: Var processors 
u\ Var inputs 
in: Var MBvec 
h: Var MBmatrix 
MBfn: Var MBcons.fn 
m,n,a,6: Var proc.plus 
prop: Var function [proc.plus — ► bool] 

T, X,Y: Var logical.clocktime 

Proof 

p.phase.commutes: Prove phase.commutes from 
phase.com -com pu te, 
phase.com .broadcast, 
phase_com_vote, 
ph ase.com .sync, 

member-phases {phases. var s. phase) 

p.initial-maps: Prove initial-maps from 
initial-da {* ♦— i@p2}, 
initial-ds {s DAmap(a)}, 
map-1 {i ♦— i®p2}, 
map_2 {i «— i@p2}, 
tnap.3 

p.phase_coin .compute: Prove phase.com .compute from 
phase.coinJx4, ph ase.com Jx 7 {i i@p1 } 

p.phase.comJxl: Prove ph ase.com Jxl from 
Jsf da {j 4— i(Q)p3}, distinct-phases, hide l 
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p_phase-com_lx2: Prove phase.comJx2 {i ♦- i@pl) from 

A/d# {s «- DAmap(s), t «- DArnap(*)), distinct-phases, map-3 

p_phase-comJx4: Prove phase_comJx4 {* t@pl} from 
phase.com Jx2, 

Ml t {5 +- DAmap(s), t <— DAmap(f)}, 

{s - S@Cy t - me }, 
map-1, 
map- 2, 
map-3, 
map-4, 

map-1 {5 *— t}, 
inap.2 {5 ♦— <}, 
map_3 {5 *— <}, 
map-4 {s «— *} 

P-phase-com-lx7: Prove phase_com_Ix7 from 

phase-com.lxl, map _7 {s 4 — <}, hide 1 1 enough-hardware 

p_pliase-Com_broadcast: Prove phase-corn -broadcast from 
corn-broadcast- J {t «— i@p3), 
coin-broadcast_2 {* <— »®p3 } , 

com -broadcast- 3 {ss «— DAmap(s), tl <— DAinap(t)}, 

com- broadcast-4 {* <— i@p3}, 

map.l {«<-«, * «— i@p2}, 

map-1 (s i, i *@p2), 

map.3 {} 

p-com.broadcast-l: Prove corn-broadcast- 1 from 
A/do , next-phase {ph *— broadcast}, distinct-phases 

p.com_broadcast_2; Prove corn-broadcast- 2 from 
com-broadcast_5, 

. 

Mis {* DAmap(s), t ♦- DAmap(t)}, 

map-1 {s ♦— s}, 

map.l {s <— <}, 

map-2 {s ♦“ a}, 

map-2 {a ♦— <} 

p-com -broadcast-3: Prove coni_broadcast.3 {1 <— t@pi} from 
A/d* {a ♦— as, t *— tl} } distinct. phases 

P-com -broadcast -4: Prove com_i)roadcast_4 from 
A/do > 

map.l {s 4 — a}, 
map.l {a <}, 

map. 3 {a a}, 

map-3 {3 4 — <}, 
map. 7 {a 4 — t}, 
distinct-phases, 
en ough -hard war c 

P-earliestJater.timc: Prove earliestJater_time from 
GOAL { p <- 1 , q — j, i «— iiu.sync.period} 
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p.elt.a: Prove elt.a from 

I * 1| {*-**}, 

j * 1| {x *- BB }, 

+1 x *2 {y «- (1 - Rho), x «- | 66 |}, 

★1 x *2 {y «- (1 - Rho), x <- |55|], 
multJeq {z 4— (1 — Rho), z 4— \BB\ t y <— \bb\] 

p.ELT: Prove ELT from 

earliest.later.time {BB <— Th — T\ , s •• — p, j <— 
elt.a {BB +— Ti — Ti }, 

*1 x *2 {x 4 - (J - Rho), y <— \bb\] 

p_ phase.com .vote: Prove phase.com. vote from 
com.vote.l {i ♦— *®p3}, 
com.vote.2 {* -« — *<0ip3}, 

com_vote.3 jss 4 — DAmap(s), tt *— DAmap(J)}, 
com.vote.4 {t ♦— i@p3}, 
inap_3 {} 

p.com.Vote.l : Prove com.vote.l from A (da , distinct. phases 

p.com.vote. 2 : Prove com_vote _2 from 
A/?, {s 4 — DAmap(s), t «— DAmap(t)}, 

Wa . 

map.l {j 4— s), 
map.l {s «— (}, 
map.2 {s 4— s}, 
map . 2 {s ♦— /}, 
map .4 {s 5}, 
map .4 {s *— t) 

p.com.vote.3: Prove com.vote.3 {i +- i©pl} from 
A/i. {5 4— ss, t 4 — */}, distinct. phases 

p.com.vote-4; Prove com.votc-4 from 
A/do , 

enough. hard ware, 
map.l {3 4 — j}, 
map.l {a t}, 
map. 3 {3 4 — 5 }, 
map.3 {3 ♦— <}, 
map_7 {3 4 — (}, 
distinct-phases 

p.phase.com.sync: Prove phase.com.sync from 
com-sync.l { 14 — t@p3), 
comjsync.2 {i 4 — i@p3}, 

com.sync-3 {33 4 — DAmap(s), It +— I)Amap(t)}, 
com_syiic.4 {}, 
map.3 {} 

p.com_sync.l: Prove com.sync_l from A/do 

p.com.sync_2: Prove com_sync_2 from 
Aft, {3 ♦- DAmap(3), t ♦— DAmap(/)}, 

ML , 

inap.l {3 4 — s), 
map.l {3 4 — <}, 
map. 2 {3 4— 3 ), 
map_2 |j4-t) 
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p.com_sync_3: Prove com-sync.3 {i «— i@pl) from A/j* {s ♦— ss f l «— M} 

p.com-syne.4: Prove com-sync_4 from 

, enough -hard ware, map_3 {a ♦— s), map_3 {5 <— /}, inap.7 {*«—*} 

End 

DA.map-proof: Module 
Using DA Jemmas, nat inductions 
Exporting all with DA Jeinmas 
Theory 

ds: Var DSstate 
da: Vnr DAstate 
k, q: Var nat 
ph: Var phases 
$ % t } x,y,z: Var DAsiaie 
p, i, j: Var processors 
u: Var inputs 
w\ Var MBvec 
h: Var MBmatrix 
MBfn: Var MBcons.fn 
Var proc.plus 

prop: Var function [proc.plus — *- bool] 

Proof 

mll_prop: function [DAstate, processors — ► function [proc.plus — ► bool]] = 

( A da, i : ( A a : 

ss.up4ate(da t a). proc(t). healthy 

= if t < a 

then da.proc(i). healthy 
else dso-proc(t). healthy 
end if)) 

nill-base: Lemma mll_prop(s, i)(0) 

mll.ind: Lemma a < nrep A mll.prop(s, i)(a) D mll_prop(s, i)(a + 1) 

p_ mil. base: Prove mil-base from 
mil-prop {da *— s, i «— *, a 0), 
ss.update {da ♦— s, k ♦— 0} 

p.mll .ind: Prove mllJnd fVom 
mil-prop {da «— s, i «— t, a ♦- a), 
mil .prop 
{ da +- s, 
i «- i, 

a - if a = nrep then nrep else a -f 1 end if}, 
ss.update {da «— s, £♦—« + ]} 

p.map.l: Prove map.l from 
DAmap {da «— s}, 

processorsJnduction {prop ♦— mll.prop(s, i), n nrep}, 

mil-prop {da 4 - s, f < — t, a «— nrep}, 

mil. base {s *— s, t <— i}, 

mllJnd {s *— s, i + — 1 , a «— m@P2) 
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ml2_prop: function [DAstate, processors — function [proc.plus -*> bool]] 

( X da, t : ( A a : 

8s_update(iia,a).proc(i).proc_state 

= if i < a 

then da.proc(i).proc_state 
else <feo.proc(t)-l>roc-*Utc 
end if)) 

m!2-base: Lemma ml2-prop(a, i)(0) 

m!2_ind: Lemma a < nrep A ml2_prop(s, i)(fl) 3 m!2_prop(a, i)(a + l) 

p_m!2_base: Prove m 12. base from 
ml2.prop {da ♦— a, i «— t, a <— 0], 
ss.updatc {da < — a, k *— 0} 

p_ml2Jnd: Prove ml2Jnd from 
ml2-prop {da «— a, t <— i, a a], 
ml2_prop 
{da ♦— 5, 
i *— *, 

a <— if a = nrep then nrep else a + 1 end if], 
ss-iipdatc {da s, k +— a -f 1} 

p_niap-2: Prove niap_2 from 
DAmap {da *— a}, 

processors_induction {prop «— inl2.prop(s, »), n nrep}, 

m!2_prop {da «— a, i *• — f, a <— nrep}, 

inl2-base {a <— a, i -• — * } , 

ml2_ind {a <— a, i -* — t, a ♦— m@P2) 

p.map-3: Prove map.3 from DAmap {da <— a} 

niM.prop: function [DAstate, processors — ► function [proc.plus — * bool]] 
( A da, t : ( A a : 

ss_update(da, a). proc(i). mailbox 
= if * < a 

then da.proc(i). mailbox 
else dsoproc(t).mai]box 
end if)) 

in 14 .base: Lemma ml4_prop(a, i)(0) 

mMJnd: Lemma a < nrep A ml4_prop(a, i)(a) 3 ml4_prop(s, *)(a + 1) 

p_ml4-base: Prove m 14 -base from 
mM.prop {da <— a, * *— i, a «— 0], 
ss.update {da ♦— a, k «— 0} 

p.niMJnd: Prove mMJnd from 
ml4_prop {da <— a, t — i f a +— a}, 
ml4_prop 
{da a, 

* ♦- «« 

a if a = nrep then nrep else a -f 1 end if}, 
ss.update {da <— a, a + 1} 


103 



p_map_4: Prove map_4 from 
DAmap {da «— s}, 

processors.induction {prop «— ml4_prop(s, i) t n «— nrop), 
mM.prop {da «— 3, * •— «, a nrep), 
m 14 -base {s <— s, i *— *}, 
m!4Jnd (s^s, » i, a «— m@P2] 

p-map_7: Prove map_7 from 
proc-extensionality 

{A «— DS(.working_setDAmap(s)), 

B «— DA(.working_sets)}, 

DSmaj-working {* <— DAmap(s)}, 

DS. working-set {s <— DAmap(s), p «— p@pl}, 

DS.working.proc {s «- DAmap(s), p «— p@pl}, 

DA.maj.working {t *— s}, 

DA. working-set {s s, p <— p@pl}, 

DA.working.proc {s s, p *— p®pl), 
map.l {1 «— p®pl } 

End 

DA_support.proof: Module 

Using DAJemmas, riat-inductions, DAJnvariants 

Exporting all with DAJemmaa 

Theory 

ds : Var DSstale 

da: Var DAstate 

kj q: Var nat 

ph: Var phases 

3 y t t T,y, z: Var DAstate 

p, i, j: Var processors 

u: Var inputs 

w: Var MBvec 

h: Var MBmatrix 

M Bfn: Var MBcons.fn 

m, n t a, b: Var proc.plus 

prop: Var function [proc.plus — + bool] 

Proof 

p.support.l: Prove support. 1 from 

proc.ex tension ality {A «— working.set(^), B «— fullset [processors]}, 
initial-da {» *— p@pl], 

DA. working-set {p p®pl}, 

DA.working.proc {p p©pl} 

p.support.4: Prove support_4 from A fd a {s <— s, t *— x) 

p.support_5: Prove support_5 from 
member.phases {phases. var <~ph], 

Afda {s <— 8, t «- X, U «- u, » «- »} 

sl 13. prop: function [MBcons.fn, processors — ♦ function[proc_plus — *■ bool]] 
( A M Bfn } i : ( A a : 

MBmatrix_cons(A/£/n, a)(t) 

= if t < a then MBfn(i) else MBmatrixO(i) end if)) 

sll3_base: Lemma sl!3.prop(A/ Bfn,i)(0) 
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sll3_ind: Lemma a < nrep A 8ll3_prop(MB/n 1 i)(a) 3 s!13_prop(MB/n, i)(a + 1) 

p.sll3_base: Prove sll3_base from 

s!l3_prop {a «— 0, i +- t}, MBmatrix.cons {k *— 0) 

p_sI13-ind: Prove sll3-ind from 
sll3_prop {o <— a, i ♦- i}, 

sl! 3 -prop {i 4 - i, a +- if a = nrep then nrep else a + 1 end if}, 

MB matrix-cons [k *— a + 1} 

p-supporl.13: Prove support- 13 from 

processors-induction {prop +— si 13_prop( M Bfn, t), n «— nrep}, 

8ll3_prop {a 4 - nrep, i i}> 
sll3.base {i 4 — i}, 
sl!3-ind {« +— i, a «— m@pl} 

p-support_]4: Prove support- 14 from 

maj-condition {A 4 — working_set(s)}, support- 1 , card_fullset 

si 15-prop: function [DAstate — ► function [nat — ► bool]] = 

(\s:(Xq: 

initial_da (a) 

3 num_good_clock8(s. sync-period, q) 

= if y < nrep then y else 0 end if)) 

sll 5-base: Lemma sll5-pxop(s)(0) 

sll 5-ind: Lemma sl15-prop(a)(y) 3 sll5-prop(s)(y -f 1) 

p_s1 15_base: Prove sll5_base from 
sllS.prop {s 4 — s f q «— 0 }, 
num -good .clocks {i 4 — 3 . sync-period, k «— 0 } 

P_sll5_ind: Prove sll 5-ind from 
sI15-prop {j s, q — q }, 
sll 5-prop {s 4 — ,f, q 4 — q + 1}, 
num -good-clocks {» 4 — s.sync_period, k 4— q 4 - 1 }, 
mitiaLda {3 *— s, * 4 — if y < nrep then q + 1 else nrep end if} 

P-support-15: Prove support- 15 from 

nat-induction {p «— sll5.prop(s), » 2 «— nrep), 

sll 5-prop (s 4 — 5 , q ♦— nrep}, 

sll 5-base {s 4 — a}, 

sllSJnd { s 4 — s, < 74 — m@pl) 

P-support-16: Prove support-16 from 
next-phase, 

prev-phase {ph 4— next .phase (ph)}, 
distinct, phases, 

member- phases {phases-var 4 — ph} 

End 

DA-broadcast.prf: Module 
Using DA.lemmas, DAJnvariants 
Exporting all with DA-lemmas 
Theory 
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ds : Var DSstate 
da: Var DAstate 
k: Var nat 
ph: Var phases 
r, s, I, x: Var DAstate 
Var DSstate 

P> V* PP* W Var processors 

«, «j, 113: Var inputs 

te: Var MBvec 

h: Var MBmatrix 

MBfn: Var MBcons.fn 

m,n,a t b: Var proc.plus 

prop: Var function [proc.pl us — ► bool] 

T, X, Y,T\,T2) BB: Var logical_docktime 

66, xx t yy, zz: Var docktime 

Tp , Sq t Rq , Rp } Epst : Var clocktiine 

int 5 : Lemma r. phase =* compute 
A reachable (r) 

A A/do (r, S, Ui ) A A/do(s, t, 1 * 2 ) A nonfaulty .clocks, r.sync.period) 

3 r.proc( 9 ).lclock € J|('W-period) 
A a . P roc(„).l<:lock€« ( * SynC - period) 

Ai. P roc( ? ).Idocke K< t9ynC - pcriod) 

pdurc: Var clocktime 
qdurc: Var clocktime 
pdurb; Var dock time 
qdurb: Var clocktime 
dur : Var clocktime 

near; function [clock time, phases — ► bool] == 

( \ duTyph : (1 — */) * duration (ph) < dur A dur < (1 + u) * duration (ph)) 

brl: Lemma r.phase = compute A A fda(r> «i ) A « 2 ) 

3 (s. phase = broadcast 
A t . phase = vote 

A s. sync, period = r.sync.period 
A l. sync.period = s.sync.period 
A ( V pp : 

nonfaulty^dock(pp, r.sync.period) 

3 clock. ad vanced (r .proc (pp) .lclock , 

s.proc(pp).lclock, 
duration (compute)) 

A dock, advanced (s.proc(pp). lclock, 

Lproc(pp)ddock t 
d uratiou (broadcast)))) 

brl a: Lemma r. phase = compute A A/d«(r, s, «j) 

3 (s. phase = broadcast 

A s.sync.period = r.sync.period 
A ( V pp : 

nonfaulty.clock(pp, r.sync.period ) 

3 clock-advanced (r . proc(pp) .lclock , 

s.proc(pp).lclock, 
duration (compute)))) 
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br 2 : Lemma r.phase = compute A A/d« (r, s, tti ) AA/Iiu(«, <, **2) 

3 3. phase — broadcast 
A i. phase = vote 

A s.sync.period = r.sync.period 
A t. sync.period = s. sync.period 

A (nonfaulty_cIock(p, r.sync.period) 

3(3 pdurc : 

near(pdurc, compute) 

A s.proc(p).lclock = r.proc(p).ldock + pdurc) 

A ( 3 pdurb : 

near(pdurb, broadcast) 

A <.proc(p).lclock = s.proc(/>).lclock + pdurb)) 

br 3 : Lemma r. phase = compute A reachable(r) A A/d«(r, s, ui ) A A /da(«> **2) 
3 3 . phase = broadcast 
A t . phase = vote 

A s. sync. period = r.sync.period 
A t. sync.period — s.sync.period 

A (noiifaulty.dock(p, r.sync.period) 

A nonfaulty.clook (g, r.sync.period) 

3 r.proc(p).ldock = r.proc(g).lclock 
A ( 3 pdurc : 

near(pdurc, compute) 

A s.proc(p).lclock = r.proc(p).lclock -f pdurc) 

A ( 3 pdurb : 

near(pdurb, broadcast) 

A t.proc(p).lclock = $.proc(p).lclock + pdurb) 
A ( 3 qdurc : 

near(qdurc, compute) 

A s.proc(g).lclock 

= r.proc(g).lclock + qdurc) 

A ( 3 qdurb : 

near(qdurb, broadcast) 

A t.proc(g).lclock 

= s.proc(g).lclock + qdurb)) 


br3.aa: Lemma r.phase = compute 
A reach able (r) 

A nonfaulty_clock(p, r.sync.period) 

A nonfaulty.clock(g, r.sync.period) 
3 r.proc(p).lclock = r.proc(g).lclock 

Proof 

p.brl: Prove brl from 
brla, 

A/da {« «- *> t t t u «2, i «- pp), 
next-phase {ph «— broadcast), 
distinct.phases 

p.brl a: Prove brla from 
A (da (s <- r, t *- s, u <- u 1 , pp), 
next-phase {ph compute) , 
distinct.phases 
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p.br2: Prove br2 

{pdurc j.proc(p).lclock - r.proc(p).lclock, 
pdurb «- t.proc(p).lclock - s.proc(p).lclock) from 
brl [pp *- p), 
clock.&dvanced 

{A' ♦— r.proc(p).lclock, 

Y *— s.proc(p).]clock, 

D «— duration (compute)}, 
clock-advanced 

{X s.proc(p).lclock, 

Y «— t.proc(p).lclock, 

D *— duration (broadcast)} 

P_br3~aa: Prove br3_aa from 

state-invariant {t *— r, da_prop <— ( A s ; ldock-eq(s))}, 

Iclock-inv, 

Iclock.eq {5 r, * 4 — p, j </} 

p-br3: Prove br3 

{pdurc 4— pdurc@pl, 
pdurb 4 — pdurb@pl t 
qdurc ♦— pdurc@p2, 

qdurb 4 — pdurb@p2} from br2, br2 {p 4 — g}, br3-aa 

br4: Lemma r. phase = compute A reachable(r) A A Ua{r y 3, ) A Afda(s t t , ti2) 
3 3. phase = broadcast 
A t. phase — vote 

A s.sync.period = r.sync.period 
A t. sync. period = s.sync.period 

A (non faulty .clock (p, r.sync.period) 

A nonfaulty_dock(g, r.sync.period) 

A Rq ^ r.proc(g).lclock 
A Rp = r.proc(p).lclock 

A Sq = $.proc(g).lclock A Tp = J.proc(p).lclock 
3(3 pdurc, pdurb, qdurc, qdurb : 
near(pdurc, compute) 

A near(pdurb, broadcast) 

A near(qdurc, compute) 

A near(qdurb, broadcast) 

A Rp = Rq 

A Sq = Rq + qdurc 

A Tp = Sq - qdurc + pdurc -j- pdurb)) 


p.br-1: Prove br4 

{pdurc 4 — pdurc@pl, 
pdurb 4— pdurbQpl, 
qdurc 4— qdurc@pl, 
qdurb •— qdurb@pl} fVom br3 
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br5: Lemma r. phase = compute A reachable(r) A -A/da(r, s, 1 * 1 ) A A/d a (s, i, U 2 ) 
3 a. phase = broadcast 
A t. phase = vote 

A s. sync.period = r.sync.period 
A t. sync.period — s. sync-period 

A ( nonfaulty ^clock (p, r.sync.period) 

A nonfaulty.dock(g, r.sync.period) 

D s.proc(,).lclock € /^sync.period) 

A t .proc(p).lclock€« ( ‘ SynC - pen0<1) 

A t.proc(p) Jclock 

> s.proc(g).lclock + duration (broadcast) 

— 2 * v * duration (compute) 

— v * duration (broadcast)) 


p_br5: Prove br5 from 
br4 

{Rq +- r.proc(g).lclock, 

Rp «— r.proc(p).lclock, 

Sq <— s.proc(g).lclock, 

Tp «— J.proc(p).ldock), 
int5, 

int5 {g +— p] 

br6: Lemma ( 3 r : 

r. phase = compute 

A reach able (r) A A/do(r, a, t»i) A s.sync.period = r.sync.period) 
AA/do(s,<,«2) 

3 a. phase = broadcast 
A t . phase = vote 

A f. sync-period = s.sync.period 

A (nonfaulty.clock(p, s.sync.period) 

A nonfaulty.clock(g, s.sync.period) 

D s.proc(j).lclock € ft<« sy n c. period, 

A t.proc(p).lclock € yj{« sync.period) 

A t.proc(p).lclock 

> a.proc(g).ldock + duration (broadcast) 

- 2 * v * duration (compute) 

— v * duration (broadcast)) 


p.br6: Prove br6 from br5 

br7: LemmaA/d 0 (i,J,t») 

3 x. phase = prev.phase(s.phase) 

A (x. phase ^ sync 3 x.sync.period = s.sync.period) 


p_br7; Prove br7 from 

support.16 {ph 4 — x. phase), 
A/'da {s *— X, t *- 8, U <- U), 
distinct, phases 


br8: Lemma reachable(s) A 5 . phase = broadcast 
D (3x,u : 

A /*«(*,*,») 

A reach able ( 1 ) A r. phase = compute A 1 . sync.period = s.sync.period) 
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pj>r8: Prove br8 {x «— a@p2, u u@p2} from 
reachable {< a), 

reachable_in_n (i +— s t k *— k@pl], 

reachable {< «— .s@p2 1 k *— if k@p\ = 0 then 0 else k€sp\ — 1 end if}, 

initial-da {5 *— 5 }, 

br7 {x $@p2, u *- «@p2}, 

prev_phase {pA 3 . phase}, 

distinct-phases 

br9: Lemma reachable(a) A A /a«(s, t, u 2 ) A s. phase — broadcast 
3 t.sync_period = $,sync_period 

A (nonfaulty_clock(p, s.sync_period) A non faulty, dock (</, s.sync-pcriod) 

3 S .proc(,).lclock € ^' s y« c -P<Tiocl, 

A t.proc(p).lcIock € tf ,synC -P eriod) 

A f.proc(p).Ic!ock 

> s.proc(g).lclock + duration (broadcast) 

— 2 * v * duration (compute) 

— v * duration(broadcast)) 

p_br9: Prove br9 from br6 {r <— x@p2, «i ti@p2}, br8 

rtpO: Lemma Sq € «<* ^sync.period) D Sq > 0 

rtpOa: Lemma T > 0 3 frame_tiine * k + T > 0 

p.rtpOa: Prove rtpOa from 

mult.non.neg {x <— frame-time, y *— k], 

★1 x *2 (x frame-time, y <— k] 

p.rtpO: Prove rtpO from 

*1 € /^* 2) {T <— Sq } t *- s. sync-period, II ^ — 0}, 

yi**) s sync_period}, 

rtpOa {T «— II®pl, k «— j.sync.period } 

rtpl: Lemma reachable(s) A A/da(s, « 2 ) A s . phase = broadcast 
3 t.sync-period = j.sync_period 
A (nonfaulty-clock(p, t. sync-period) 

A nonfaulty-clock(</, s.sync_period) 

A enough_clocks(s.sync_period) 

A Tp = f .proc(p).lclock 
A Sq = j.proc(g).lclock 
A Epai 

= 2 * v * duration (compute) + u * duration(broadcast) 
A duration(broadcast) — Epsi > 0 

d r /; sync - period) (T P ) 

> re (- *ync-period) (S9) 

+ (1 — Rho) * |duration (broadcast) — Epsi\ 

-*) 


p-rtpl: Prove rtpl from 
rtpO, 
br9, 

ELT 

{da 4 — a , 

Ti - Tp, 

T, - Sg, 
q <- q> 

bb 4 — duration (broadcast) — Epsi) 
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rtp2: Lemma reachable ( 5 ) 

AMda{s 1 t i u 2 ) 

A 3. phase = broadcast 

A nonfaulty.clock(p, s.sync.period) 

A nonfaulty_clock(g, s.sync.pcriod) 

A enough .clocks (s. sync_period) 

A Tp = Lproc(p).lclock 
A Sq = s.proc(g).lclock 
A Epsi 

= 2 * v * duration (compute) + v * duration (broadcast) 
A duration (broadcast) — Epsi > 0 

3 r <<,' 8ync - period) (T P ) 

> r* ( , , ' synC ' , ' er '°^(Sg) + (1 — Rlio) * |duration(broadcast) — 

p.rtp2: Prove rtp2 from rtpl 

rtp3: Lemma reachable (s) 

AAfda(s,t,U2) 

A s . phase = broadcast 

A nonfaulty_clock(p, s.sync.period) 

A nonfaulty_clock(tf, s.sync.period) 

A enough-clocks (j. sync.period) A i.sync.period = s.sync.period 
Drt Osync.period) (<proc(p)|dock) 

> rt ( ,‘ Sync ' per ' 0< ^ ) (i.proc( 7 ).lclock) + inax.comni.delay 

p_rtp3: Prove rtp3 from 
rtp2 

{Epsi «- 2 ♦ v * duration (compute) + v * duration (broadcast), 

Sq *— s.proc(^).lclock, 

Tp ♦— i.proc(p).lclock}, 
broadcast.duration, 
broad cast_duration 2 

rtp 4 : Lemma reachable(s) 

A A fda(Sj t, ti'i) 

A 3. phase = broadcast 

A non faulty. clock(p, s.sync.period) 

A nonfaulty_clock(g, s.sync.period) A enough. clocks (s.sync.period) 

3 da_rt{/, p, *.proc(p).lclock) 

> da.rt(s,?, s.proc(g).lclock) + max .comm .del ay 

rtp4a: Lemma reachable(s) A A/d«( 3 , t, « 2 ) A s . phase = broadcast 
3 f.sync.pcriod = s.sync.period 

p.rtp4a: Prove rtp4a from 
A fda {$ «— s, / «- t, u <— U 2 }, distinct. phases 

rtp 4 b: Lemma reachable(s) AA/rfo(s»^»«) D reachable(f) 

p_rtp4b: Prove rtp4b from 
reachable {k «— A;@p3}, 
reachable {t *— s), 

reach able.in.n {k «— k@p2 + 1, s +— s, u +— u) 
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p.rtp4: Prove rtp4 from 
rtp3, 

rtp4b {« ti^}, 

rtp4a, 

da.rtJem {da «- (, p «- p, T *- f.proc(p).lclock), 
da.rl.lem {da «— s t p g, T *— s.proc(g).!doek} 

rlp5: Lemma reax hable(s) 

/\Afda(3,t f u) 

A 3 . phase = broadcast 
A 5. proc(p). healthy > 0 

A b r oad cast. received (5 1 1 , p) 

A(Vg: 

s. proc(g). healthy > 0 

3 da_rt(s, g, s.proc(g).lclock) -f in ax .comm -del ay 
< da_ri(i, p, (.proc(p).lclock)) 

3 broadcast.rceeived(DAmap(s), DAmap(t) f p) 

p_rt|>5: Prove rtp5 {g <- gg®p’2} from 
distinct.phases, 

I)S. broadcast. received {a *- DAmap(s), t <— DAmap(i), gg ♦- g}, 

DA.broadcast.received {gg «— gg@p2}, 

map.] {5 j, t «— gg®p2}, 

map_4 {s ♦- a, i ♦— gg®p2}, 

niap.4 {s <— (, 1 4 — p } } 

A/a<» 

rtp6: Lemma reachable(j) A a. proc(p). healthy > 0 
3 nonfaulty .clock (p, J.sync.period) 

p_rtp6: Prove rlpf> from 
nfclk.inv, 

state-invariant {( <— s, da_prop <— ( A 3 : nf.clks(s))}, 
nf.clks {» ^ — p} 

rtp7: Lemma reachable(s) A A/j 0 (a, t, u) A s . phase = broadcast 
3 enough-clocks(s.sync.period) A (.phase — vote 

p_rtp7: Prove rtp7 fVom 
Ado > 

stateJnvariant {da_prop ♦— ( A s : enough. hardwarc(s)), ( +— s}, 
enough.inv, 

enough-hardware {( «— s} t 
next.phase {ph ♦— a. phase], 
distinct.phases 

p.com.broadcast_5: Prove com_broadcast_5 from 
rtp4 {u 2 «— «, q *- g@p2, p <— i}, 
rtp5 {p — i), 
rtp6 {p 1 - *}, 
rtp6 {p «— g®p2}, 
rtp7 

End 

DAJntervals: Module 
Using DA .broadcast, prf 
Exporting all with DAJemmas 
Theory 
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ds: Var DSstate 

da: Var DAstate 

k: Var nat 

ph : Var phases 

r, j, f, r: Var DAstate 

js, Var DSstate 

p, g, pp, qq> Var processors 

u, Var inputs 

to: Var MBvec 

h: Var MBinatrix 

MDfn : Var MBcons_fn 

m, n, a, 6: Var proc.plus 

prop: Var function[proc_plus — ► bool] 

T, A r , Y, T\ , T 2 , BB: Var logical.clocktime 

bb i XT,yy,zz } T 2 i yr- Var clock lime 

Tp, i?psi: Var clocktime 

pdurc: Var clocktime 

qdurc: Var clocktime 

pdurb: Var clocktime 

qdurb: Var dock time 

dur: Var dock time 

Proof 

br.int: Lemma r.phase = compute A reachable(r) A A/do(r, s, «i) A U 2 ) 

3 s . phase = broadcast 
A t , phase = vote 

A 3 . sync-period = r.sync-period 
A t. sync-period = s. sync-period 

A (non faulty . clock (g, r.sync-period) 

3(3 qdurc, qdurb : 
near(qdurc, compute) 

A near(qdiirb, broadcast) 

A s.proc(<y).lclock = r.proc(r/).ldock + qdurc 
A f.proc(?).lclock = s.proc( 9 ),ldock + qdurb)) 

p.brJiit: Prove brJnt {qdurc <— qdurc@pl, qdurb qdurb@pl} from 
br3 {p «- q] 

inlO: Lemma r.phase =* compute 
A reach able (r) 

A A&.(r, s, tii ) A A r da(3, t,u 2 ) A nonfaulty_clock(g, r.sync-period) 

3 r.proc(g).ldock = r.sync_period * frame-time 

Ar.proc(«,).lclock€/f (r8yn< '- period) 

p-int(): Prove inti) from 
clkvaLinv, 

state-invariant {da_prop *— ( \r : lclock_val(r)), t *- r}, 

Idock-vaJ {i <— q, 3 «— r}, 

*1 € /^* 2) {T *- r.proc(</).Iclock, i r.sync-period, II «- 0}, 

2 <*i) r.sync-period} 
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inti: Lemma r.phase — compute A reachable(r) A jVd 0 (r, s, ) A A/ja(s, ** 2 ) 
5 . phase = broadcast 
A /. phase = vole 

A s.sync.period = r.sync.period 
A /. sync-period = j.sync.period 

A (non faulty .clock(^, r.sync.period) 

D r.proc(g).ldock = r.sync.period * framc.time 
A r.proc(f).klock 6 R^ync. period, 

A 3 .proc(,).lclock e /;< - sync. period)) 

intla: Lemma ix < yy A yy < zz 0 xx < zz 

p.inlla: Prove intla 

p.intl: Prove inti from 
into, 
brJnt, 

*1 € A ( * 2) 

{T ♦— s.proc(g).lclock, 
i *— s. sync.period, 
n qdurc@p2}, 

7 <*J) ^.sync.period}, 

pos.du rations, 
all.du rations, 
intla 

{zz qdurc^p2, 

yy *— (1 — v) * duration (compute), 
xx *— 0} , 
intla 

(xz «— qdurc@p2, 
yy «— (1 + y) * duration (compute), 
zz «— frame.time} 

inti: Lemma r.phase = compute A reachable (r) A A/d a (r, a, t*i ) A A/da(a, t, « 2 ) 
Z> s . phase = broadcast 
A /.phase = vote 

A a. sync.period = r.sync.period 
A / .sync.period = 5. sync.period 

A (nonfaulty_clock(</, r.sync.period) 

A r.proc(g).lclock = r.sync.period * frame.time 

3 f.proc(g). [clock € ^ nc -P etiod) ) 

int2a: Lemma near(qdurc, compute) A near(qdurb, broadcast) 

3 0 < qdurc + qdurb A qdurc -h qdurb < frame.time 

p.int2a: Prove int2a from 
pos.du rat ions, 
all.du rat ions, 

★ 1 x *2 {x <— (] - y), y *— duration (compute)}, 

★ 1 x *2 {x <— (1 — u) 1 y *— duration (broadcast)} 

p.inti: Prove int2 from 
j(*0 1 1 +_ /.sync.period}, 
br.int, 

*1 € « ( * 2) 

{T «— /.proc(<y).lclock, 
i <— /.sync.period, 

FI <— qdurc®p2 + qdnrb@p2}, 
int2a {qdurc «- qdurc@p2, qdurb qdurb©p2} 
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int3: Lemma r. phase = compute A reachable(r) A jS/^ rt (r, s, ) A j\/ii a (s, /, u-j) 

3 (nonfaulty-dock(g, r.sync_period) 

3 r.proc(g).lclock = r.sync-period * frame-time 

A r.proc(?).lclock € tf' s y" c -P erio < i > 

A s.proc(g).lclock € ^(» sync-period) 

A<.proc(,).lclock€«“ SynC - pcriod) ) 

p_int3: Prove int3 from inti, int2 

int4: Lemma (r.phase = compute 
A reachable (r) 

A Mda(r, 3, m) A Afda(3, t, u 2 ) A nonfaulty_clock( 9 , r.sync.period)) 
3 (r.proc(g).lclock = r.sync.period * frame-time 
Ar.proc(,).lclock € fl(' s .ync-period) 

A .9.proc(fl).lclock € s y nc -P er * od > 

A t.proc(,).lclock € «f«-*ync-Pcriod»j 

p_int4: Prove int4 from int3 
p int5: Prove int5 from int4 

End 

elk-types: Module 
Exporting all 
Theory 

realtime: Type Is number 
JogicaLclocktime: Type is number 
physical-clock time: Type is number 
clocklime: Type is number 
x: Var number 

posnum: Type from number with ( A x : x > 0) 
po 9 -logical_clocktime: Type is posnnm 
posrealtime: Type is posnum 

fraction: Type from number with (Az:l>xAx>0Ax^l) 
period: Type is nat 

End 

clkmod: Module 

Using rcp.defs, absmod, clk.types 

Exporting all with rcp_defs,dk-types, absmod Theory 
posreaJtime 

E, A: pos JogicaLclocktime 

frame-time, sync_time: posJogical.clocktiine (* Changed from R, S *) 
i: Var period 
k: Var nat 

7^°: logical, clocktime *s= 0 

function [period — ► logical-docktime] = ( A i : T° -f * * frame.timc) 
T-next: Lemma 7^* +1) = 7^** + frame.time 
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T, II: Var logical.clocktime 
Tj,T 2 ,To,Tn: Var physicaLclocktime 

★1 € function [logical- clocktime, period — ♦ boolean] = 

(AT,t:(3IT:0<nAn< frame-time A T = 7^’) + II)) 

★ 1 6 function [logical- clock time, period — »■ boolean] = 

( A T, t : ( 3 II : 

0 < II A II < sync-time AT = 7**> -f frame.tiine - sync-tirne -|- II)) 
p, q y t : Var processors 

c*j(*2): function [processors, physicaLclocktime — *• realtime] 
log.to-phys: function [logical-clocktime physical, clock time] == 

( A T — * ► physicaLclocktime : T) 
x: Var number 

function [number — ► number] == ( A x : x/2) 
p : fraction 
Rho: fraction = ^ 

gooddock: function [processors, physical.clocktime, physical.clocktime 
— ► bool] = 

( A p, To, Tn ’ 

(VTi f 7i : 

To < T x A To < T 2 A Tj < T n A 7 2 < Tn 

D |c r (Ti ) - c P (T 2 ) - (Ti - T 2 )| < Rho * |T, - T a |)) 

monotonicity: Theorem 

( 3 To, 7 n - gooddock (p, To, Tn) A To <T\ A To < T 2 A T\ < Tn A 7 2 < Tn) 
D(T, >T 2 Dc p (T) f >c p (T 2 )) 

A^* 2 ^: function [processors, period — ► clocktime] 

(♦ aean of the skews within tolerance *) 

Delta2: function [processors, processors, period — ♦ clocktime] 

(* Measured skew *) 

initial.Corr: function [processors — * clocktime] == (Ap-+ number : 0) 
second.arg: f unction [processors, period — ► nat] == ( A p, » : ») 

Corr** 2 * : Recursive function [processors, period — ► clocktime] = 

( A p, i : if i > 0 

thenC’orr},P red < < » + A ( p P red,, » 
else initial.Corr(p) 
end if) by second.arg 


i4^* 2 ^(*3): function [processors, period, logical. clocktime 

— ► physical.clocktime] == ( A p, 1 , T : T -f Corr p ^) 
r/^j 2 ^(^3): function [processors, period, logicaJ.clocktime — ► realtime] = 
(A p,i,T:c p (A^(T))) 

skew: function [processors, processors, clocktime, period — ► clock time] == 
( A p, g, T, 1 — ► clocktime : |r< p ^(T) — rt^(T)|) 


non faulty. clock: function [processors, period — ♦ boolean] = 
( A p,i : gooddock (p, j 4{, 0) (T (0, ) 1 /lJ,' ) (7’ <,+,) ))) 
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mini-measure: function [period, nat — * nat] == ( A i>k : k) 
num-good_clocks: Recursive function [period , nat — ♦ nat] = 
( A i, ib : if A: = 0 V t > nrep 
then 0 

eisif nonfaulty_clock(A:, *) 

then 1 4- num-good_clocks(i, k — 1) 
else num_good_clocks(t, k — J) 
end if) by num -measure 


enough-docks: function [period — ► bool] = 

( A i : 3 * num-good_clocks(* 1 nrep) > 2 * nrep) 


SlA: function[pcriod — ► bool] == ( A i : enough-docks(i)) 

(* in current clock sync theory ■ 

(LAMBDA i : 

(FORALL r : (m + 1 <■ r AND r <- n) IMPLIES nonf aulty_clock(r , i))) 

♦ ) 


SIC: function [processors, processors, period — ► bool] = 

( A P. ?. » : ( V T : 

nonfaulty.clock(j), «) A nonfauUy.clock(g, i) A 7’ € /c‘ 

D skcw(p, q, T , ») < />)) 

SlCJemma: Lemma SlC(p, q, i) DSlC(j,p,«) 

Si : function [period — * bool] = ( A « : SlA(i) 3 (Vp,g: SlC(p,j,*))) 

function [processors, period — ► bool] = 

( A p, » ; (|Corr[> ,+1) - Corral < £)) 

(♦ The following three theorems were proved in the clock sync theory. 
They are taken as axioms here. *) 
adj-always-pos: Axiom A^p\T^ k ^) > T° 

Theorem-1: Axiom Si (i) 

(* THEOREM *) 

Theorem_2: Axiom S 2 (p, ») 

(* THEOREM O 

Aft: Axiom skew(p, q, T*°\ 0) < So 

Al: Lemma non faulty _dock(p,i) = gooddock(p, Ap°\T* 0 ^), Ap ^(X* ,+l ^)) 

A2: Axiom nonfaulty_clock(p, t) 

A nonfaulty_clock(g, t) A Si C(p, g, i) A 52 (p, *) 

3 K'>i < sync_time 

A ( 371, : 7o € 5<’> A |rtp ) (7o + A$) - rt['\T 0 )\ < <r) 

A2.aux: Axiom App = 0 

m: processors (* maximum number of faulty clocks *) 

CO: Axiom m < nrep A m < nrep — num_good_docks(i, nrep) 

Cl: Axiom frame-time > 3 * sync-time 
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C2: Axiom sync_lime > E 
C3: Axiom E > A 

C4: Axiom A>6 + e+ |* sync-time 
C5: Axiom 8 > S Q 4 P * frame-time 

C6: Axiom £>2*(f4p* sync.time) + 2 * m * A/(nrcp - m) 

4 nrep * p * fraine.tiinc/(nre|> — m) 

4- p * A 

4 nrep * p * E/(nrep — w) 

sync-thm: Theorem 
enough-docks (i) 

D(Vp,?; 

(VT : nonfaulty-dock(p, i) A noiifau]ty.dock(fl, t) AT € IC'’ 

D |rtp>(T) - r<</>(T)| < «)) 

Proof 

p_sync_thm: Prove sync_thin from 
Theoreni-1 {t «— *}, Si {i «— i}> SIC {i *— s} 

End 

dkprop: Module 

Using elk mod, DA 
Exporting all 
Theory 

T, Tj , T 2 , T 3 , T 4 , BB, 7o, Tn,TX, TY: Var logicaLclocktime 
p, q: Var processors 
da: Var DAstate 
i: Var period 

ft2: Lemma good dock ( q, T° , T\ 4 BB) A (7\ > 7^) A ( BB >T°) 

3 |c<,(T, 4 BD) - c,(7i ) - ii£| < Rho * \BB\ 

ft3: Lemma goodclock^T 0 ,^ 4 BB) A (7i > T°) A {BB > 7°) 
3(1— Rho) * \BB\ < c q {% 4 BD) - c,(7i) 

A c,(Ti 4 - Cq(Ti) <(14 Rho) * \BB\ 

ft4: Lemma enough-docks (i) 

A n on faulty .dock (p, i) A nonfau]ly_c]ock(g, «) A T € 

3 -S < r t ( ;\T) - rtq'\T) A rl\P(T) - r i<°(T) < « 

ftS: Lemma gooddock(g, 7 10 , Tj + Corrl,' 1 + BB) 

A (T, > T°) A (T, + Cott^ > 7 10 ) A (BD > 7°) 

3 (1 - Rho) ♦ \BB\ < rtf(Ti + BB) - r ^(T, ) 

A r4°(7i + 55) - rt^(Ti ) < (1 + Rho) ♦ \BB\ 

ft6: Lemma T 2 = T\ 4 BB 

A good dock (g, 7 10 , T] 4 Cort ^ 4 BB) 

*(^> 7 °) 

A (T, + Corr* 0 > T°) 

A (BB > T°) 

A enough-docks(t) 

A nonfaulty-dock(p, i) A nonfaiiIty-dock(g, i) A T 2 € 
3 rt ( ;>(T 2 ) > r lW(T, ) + ( l - Rho) * \BB\ - 6 
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fl7: Lemma T 3 < T t A %oodc\ock(q,T° ,1\) 3 good clock ( g, T* 1 ,T 3 ) 

ft8: Lemma T x + BB < 7^‘ +1) A nonfaulty.clock(?,i) 

3 goodclock(g, T°,Tj + Corr,** + BB) 

fl9: Lemma Tj = T X + BB 
aTj < T<’ +1) 
a(Ti >7°) 

A(T, + CorTq'^ > T°) 

A (BB >T°) 

A enough-docks(t) 

A nonfaulty_clock(p, i) A nonfauUy.dock(?, t) A r i\ € rt (t) 
D r< ( p 0 (T 2 ) > rt ( ,°(Ti) + (1 - Rho) ♦ \BB\ - t 

ftlO: Lemma Tj € 7£^ 3 Ti < T*' +1 ' 

fill: Lemma T 2 = Ti + B B 
A (T, >T°) 

A (Ti + Corr^ 0 > T°) 

A(BB>T°) 

A enough-docks (i) 

A nonfaulty_dock(j>, t) A nonfaulty_clock(g, i) A 7 2 € R ^ 

3 rtp ) (Tj) > rt ( ;>(T,) + (1 - Rho) * \BB\ - S 

ft] 2: Lemma Ti € /£*’* 3 (T\ + Corr ^ > 7^*) 

GOAL: Lemma T 2 = Tj 4- BB 
A (Ti > T°) 
a(/JZ? > 1°) 

A non faulty -dock (p, *) 

A nonfaulty_dock(</, i) A enough_clock8(t) A T 2 € itf** A Ti € 
3 4°(T 2 ) > rl^tTi) + (1 - Rho) ♦ \BB\ - 6 

nfcJem: Lemma nonfaulty-dock(p, i -f 1) 3 nonfaully-dock(p, i) 

Proof 

nfc-a: Lemma + Corr p ) < T* ,+2 ^ + Corrp* 1 ^ 

P-nfc_a: Prove nfc_a from 
7^* J) {t »+ t}, 

7 **° {* — * + 2 ), 

Theorem _2 {i «— * } , 

S2{s-i}, 

abs.main {x «— Corr p +1 ^ — Cortp\ z <— S), 

Cl, 

C2 

p-nfcJem: Prove nfcJem from 
iionfauUy.clock, 
nonfaulty_cIock {t <— i + 1}, 
good clock 

{T n ~ T (,+2) + Corr^ +1) , 

To — 7 <0) + Corr ( p °\ 

Ti «- T,® P 4, 

T 2 -T 2 ®p4). 

goodclock {Tn - T< ,+1) + Corrj, 0 , T 0 — 7< 0) + Corrj, ) }, 
nfc_a 
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pJTt2: Prove ft2 from 
goodclock 

{p — 7. 

T 0 -T°, 

TV «— 7i + BB, 

T 2 -Tl 

7i - (T, + BB)} 

P_ft3: Prove ft3 from 
ft2, 

absJeq {x - c q {T x + BB) - c,(T,) - BB , * «- Rho * 
abs_geO {x <— BB} 

p„ft4: Prove ft 4 from 

sync-thm {i ♦- »} , absJeq {x «— ri^(T) — rf^(7*), z <— />} 

p.ft5: Prove ft5 fVom 
ft3 {T, «- Ti + Corr' 0 }, 
rt ( *t J) (*3) {p 7. T *-Ti, t < — * } » 
rtf '(*3) {p *- ?, T *-Ti + BB, i *- ,} 

p_ftf»: Prove ft6 from ft4 { T -• — T 2 } > ft5 

P_ft7: Prove ft7 from 
goodclock 
{p — 7 . 

To *- T°, 

T, «- Ti @p2, 

Tj - Tj© p2, 

Tw «- Tt }, 

goodclock {p *- q, To «— T°, Tn <— 7 3 ) 

ft8a: Lemma Corr ^ = 0 

p_ft8: Prove ft8 from 
nonfaulty.clock {p «— q t i i), 

fl7 m <— Ti + Corr^ 0 + BB, T« - T< i+1 > + Corr ( , 0 }, 

T** 1 ' {1 v- 0}, 
ft8a 

p.ft8a: Prove ft8a from G’orrJ,* 2 ' {» *— 0, p •— ?} 
p_ft9: Prove ft9 from ft6, ft8 
p.ftlO: Prove ft 10 from 

*1 € I&* 7 ^ {T *— T *2 , II <— fraine.timo}, ' 1 {i ♦— i}, 7** 1 * {t <_ * + ]] 
■p-ftll: Prove ftll from ftlO {i <— »}, ft9 
P_ftl2: Prove ft 12 from 

adj_aJways_pos {k * - i, p <- </}, *1 € I&* 7) [T «- Ti, i <- i} 
p_GOAL: Prove GOAL from ftll, ftl2 

End 

DAJnvariants.tcc: Module 
Using DA jnvariants 
Exporting all with DAJnvarianta 
Theory 


120 



it: Var naturalnumber 
p : Var rcp_defs. processors 
j: Var rcp_defe. processors 
i: Var rcp_de£s. processors 
x: Var DA.DAstate 
n i: Var naturalnumber 
k: Var naturalnumber 
u: Var rcp.defs. inputs 
t: Var DA.DAstate 
s: Var DA.DAstate 

CorrJein_TCCl: Formula (*« > 0) 3 (it — 1 > 0) 

Proof 

Corr.lem.TCC 1 _P ROOF : Prove Corr.lem.TCCl 
End DAJnvariants-tcc 
DA-map-proof-tcc: Mddule 
Using DA-map-proof 
Exporting all with DA_map_proof 
Theory 

a: Var rcp-defs.proc_plus 
p: Var rcp.defs. processors 

m: Var rcp-defs.proc_plus p_mll_base_TCCJ : Formula ((0 > 0) A (0 < nrcp)) 

p.nill Jnd-TCCl: Formula 

(( if a = nrep then nrep else a + 1 end if > 0) 

A ( if a = nrep then nrep else a + 1 end if < nrep)) 

p_map-l_TCCl: Formula ((nrep > 0) A (nrep < nrep)) 

Proof 

p-inll-base-TCCl -PROOF: Prove p_inll_base_TCC] 

P-mll Jnd-TCCl -PROOF: Prove p_mll_ind-TCCl 
P-inap-l-TCCl-PROOF: Prove p_inap_l-TCCl 
End DA_map-proof_tcc 
DA_support_proof_tcc: Module 
Using DA_support_proof 
Exporting all with DA_support_proof 
Theory 

q: Var naturalnumber 
a: Var rcp-defs.proc-plus 
m : Var naturalnumber 
w: Var rcp_defs.proc_plus 
p: Var rcp-dcfs. processors 

P-sl 1 3_base_TCO 1 : Formula ((0 > 0) A (0 < nrep)) 

p_sl 1 3 Jnd_TCC 1 : Formula 

(( if a = nrep then nrep else a + 1 end if > 0) 

A ( if a = nrep then nrep else a -f 1 end if < nrep)) 
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p_support_ 1 3.TCC 1 : Formula ((nrep > 0) A (nrep < nrep)) 

p_»ll 5 JjruLTCCl : Formula 

(( if q < nrep then q + 1 else nrep end if > 0) 

A ( if q < nrep then q + 1 else nrep end if < nrep)) 

Proof 

p_sl 1 3-base_TCC 1 _P ROOF: Prove p_sll3_base_TOCi 
p_sll3-ind-TCCl -PROOF: Prove p_s!13Jnd_TCCl 
p_su pport_l 3.TCC I .PROOF : Prove p_support_33_TCCl 
p-sll5jnd-TCCl-PltOOF: Prove p_sll5Jnd_TCCl 
End DA_supporl_prooLtcc 
DA-to-DS-tcc: Module 
Using DA_to-DS 
Exporting all with DA_to_I)S 
Theory 

da: Var DA.DAstate 
s: Var DA.DAstate 
t; Var DA.DAstate 
k: Var naturalnumber 

MBfn : Var function [rcp-defs. processors rcp-defs.MBvec] 

88.iipdate.TCCl: Formula (->((Jb = 0) V (jb > nrep))) D (k - 1 > 0) 

88.update.TCC2: Formula (->((Jb = 0) V (k > nrep))) D ((fc > 0) A (k < nrep)) 

ss_update_TCC3: Formula 

(-.((j t = o) V (k > nrep))) 3 da-measure(da, k) > da-measure (da, k - 1) 

MBmatrix-cons-TCCl: Formula 
H(* = 0) V (k > nrep))) 

3 MBmc-ineasure(Af B/w, k) > MBmc_measure(A/B/n, * - 1) 

reachable-in-n_TCCl: Formula (->(k = 0)) 3 {k - 1 > 0) 

reach able_in_n-TCC2: Formula 

(-»(* =S 0)) 3 da_measure(t, k) > da-ineasure(s, k - 1) 

Proof 

ss_update_TCCl_PROOF: Prove ss_update_TCCl 
ss_update_TCC2_PROOF: Prove ss_updale-TCC2 
ss-update-TCC3-PROOF: Prove ss_update_TCC3 
MBmatrix-cons-TCCl-PROOF: Prove MBmatrix_cons_TCCl 
reachable-in-n-TCCl_PROOF: Prove reachable_in.n_TCCl 
reachable-in_n-TCC2_PROOF: Prove reachableJn_n_TCC2 
End DA-to-DS-tcc 
DA-tcc-proof: Module 

Using clk-types-tcc, clkmod.tcc, DA_ntap_proof_tcc, DA_support_proof_tcc 
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Exporting all 

Theory 

Proof 

posiwm-TCCl- PROOF: Prove posnum-TCCl {* «- 1} 
fraction_TCC 1 .P ROOF : Prove fraction-TCCI {x <- 0} 

C6.TCC1-PROOF: Prove C6.TOC1 from CO 

p sll5-ind-TCCl-PROOF: Prove p_sll5_ind..TCCl from proccssors_exist_ax 
Rho.TCCl -PROOF: Prove Rho.TCCl (♦ needs printerpdivide - yes •) 


End 

DA-broadcast-prf-tcc: Module 
Using DA-broadcast-prf 
Exporting all with DA-broadcast-prf 

Theory 

i: Var rcp_defs. processors 
q: Var rcp-defs. processors 
qq : Var rcp-defs. processors 
IT: Var number 
r: Var DA.DAstate 
fc: Var naturalnumber 
u: Var rcp.defs.inputs 
s; Var DA.DAstate 
qdurb: Var number 
qdurc: Var number 
pdurb: Var number 
pdurc: Var number 

p.br8-TCCl: Formula ( if Jfc = 0 then 0 else k - 1 end if > 0) 
Proof 

p.br8-TCCl -PROOF: Prove p-br8-TCCl 
End DA-broadcast_prLtcc 

dk.ty pcs. ice: Module 
Using elk-types 
Exporting all with elk-types 
Theory 

z: Var number 

posnum-TCCl: Formula (3i : x > 0) 
fraction.TCCl: Formula (3*:1>*A*>0A*/1) 

Proof 

posnum-TCC l.P ROOF : Prove posnum.TCCl 
fraetion-TCCl. PROOF: Prove fraction-TCCI 
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End clk_types_tcc 
dkmocLtcc: Module 
Using dkmod 
Exporting all with dkmod 
Theory 

*: Var naturalnuinber 
k : Var naturalnuinber 
p: Var rcp_defs. processors 
halLTCCl: Formula (2 ^ 0) 

Rho.TCCl: Formula (1>|A|>0A|^1) 

Corr.TCCl: Formula (i > 0) 3 second _arg(p, i) > second_arg(;j l pred(i)) 

num_good_clocks_TCC1: Formula 

(-«( k — 0 V k > nrep)) 3 (( k > 0) A (k < nrep)) 

num_good.docks.TCC2; Formula 

(nonfauIty.dock(&, t)) A = 0 V k > nrep)) 3 {k — l > 0) 

num.good_docks.TCC3: Formula 

(“»(nonfauUy_dock(£, i))) A = 0 V t' > nrep)) 3 (Jt — 1 > 0) 

num.good_clocks.TCC4: Formula 

(nonfaulty_clock(l: # *)) A (->(k = 0 V k > nrep)) 

3 num_measure(i\ jfc) > num_rneasure(t, k — 1) 

num.good.clocks.TCC5: Formula 

(-'‘(nonfaulty_clock(fc, i))) A (->(£ = 0 V k > nrep)) 

3 num_measure(i t k) > num_measure(t, k — l) 

C6.TCC1: Formula ((nrep — m) ^ 0) 

Proof 

half.TCCl. PROOF: Prove half.TCCl 
Rho.TCCl.PROOF: Prove Rho.TCCl 
Corr.TCCl _P ROO F: Prove Corr.TCCl 

num_good.docks_TCCl .PROOF: Prove num_good.docks.TCCl 
num_good.docks_TCC2.PROOF: Provo num.good_docks.TCC2 
num.good.clocks_TCC3-PROOF: Prove nuin.good_docks.TCC3 
num.good_docks_TCC4_PROOF: Prove num_good.docks.TCC4 
num_good.docks.TCC5_PROOF: Prove num_good_clocks_TCC5 
C6.TCC1.PROOF; Prove C6.TCC1 
End clkmod.tcc 
lop: Module 
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Using rcp.defs, generic.FT, sets [processors], cardinality [processors], 

nat -inductions, noethenan[proc_plus,lessp], US, RS, RS.majority, RS.to-US, 
RS Jemmas, RS.in variants, RS-top-proof, RS.tcc.proof, rcp.defs.tcc, 
RS-to.US.tcc, DS, DS-to-RS, DSJemmas, DS.top-proof, DS.map-proof, 
DS_support-proof, multiplication, absmod,clk_types, elk mod, DA, DA-to_DS, 
DAJn variants, clkprop, DA Jemmas, DAJop-proof, DA.map-proof, 
DA-Support-proof, DA.broadcast.prf, DA Jntervals, rcp.defs-tcc, 
DS-to-RS-tcc, DS-support-proof.tcc, DS.map.proof.tcc, DAJnvariants-tcc, 
DA-map_prooLtcc, DA-support.prooLtcc, DA_to_DS-tcc, clk-typesJcc, 
clkmod-tcc, DA_tcc-proof, DA_broadcast.prf.tcc 

Theory 

u: Var inputs 
usl,us2: Var Pstate 
rsl,rs2: Var RSstate 
dsl, ds2: Var DSstate 
dal , da2: Var DAstate 

RS_f rame.comm u tes : Theorem 

reach able (rsl) A jV r ,(rsl, rs2, «) D M. s (RSmap(rsl), RSmap(rs2), u) 

RSJnitiaLmaps: Theorem initiaLrs(rsl) D initial.us(RSmap(rsl)) 

DS-frame_commutes: Theorem 

dsl .phase = compute A frame_N_ds(dsl , ds2, u) 

3 A/i-«(DSinap(dsl), DSmap(ds2), u) 

DSJnitiaLmaps: Theorem initiaJ.ds(dsl) 3 initiaLrs(DSmap(dsl )) 

DA-phase.commutes: Theorem 

reach able (dal) A A/a u (dal , da2, u) 3 A/di(DAmap(dal), DAmap(da2), tt) 
DAJnitiaLrnaps: Theorem itiiliaLda(dal) 3 initial_ds(DAmap(dal)) 

Proof 

p_RS-frame_commutes: Prove RS-frame.commutes from 
RS_to-US.fr ame.com mutes {s «— rsl, t «— rs2} 

p.RSJnitiaLmaps: Prove RSJuitiaLmaps from 
RS_to-US. initial. maps {s <— rsl} 

p-DS.frame-commutes: Prove DS-frame.com mutes from 
DS-to-RS. frame.com mutes {s <— dsl, t ♦— ds2) 

p-DSJnitial.maps: Prove DSJnitial.maps from 
DS.to-RS.initial.maps {s <— dsl} 

p- DA -ph ase.com m u tes: Prove DA.phase.commutcs from 
. DA-to-DS.phase.commutes {s <— dal, t da2) 

p.DAJnitiaLmaps; Prove DAJnitial.maps from 
DA-to_DS. initial- maps {s <— dal] 

End 

rep-defs: Module 
Exporting nil 
Theory 
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p : Var nat 

Fstate: Type (* computation state of a single processor *) 
inputs: Type (* type of external sensor input *) 
outputs: Type (* actuator output type *) 

MB : Type (* mailbox exchange type *) 

nrep: nat (* number of replicated processors *) 

initiaLproc_state: Pstate (* assumes each processor begins identically *) 
recovery-period: nat (* number of healthy frames required to recover 

from transient fault plus one •) 
recovery-period-ax: Axiom recovery-period > 2 

processors-exisl_ax: Axiom nrep > 0 

processors: Type from nat with ( A p : (p > 0) A (p < nrep)) 

MBvec: Type = array [processors] of M B 
MBmatrix: Type = array [processors] of MBvec 
phases: Type = (compute, broadcast, vote, sync) 
ph: Var phases 

next.phase: function [phases — ► phases] = 

(A ph : If ph = compute 
then broadcast 

elsif ph = broadcast then vote elsif ph = vote then sync else compute 
end if) 

prev- phase: function [phases — ► phases] = 

(A ph : if ph = compute 
then sync 
elsif ph = broadcast 
then compute 

elsif ph as vote then broadcast else vote 
end if) 

proc.plus: Type from nat with ( A p : (p > 0) A (p < nrep)) 

ft, 171,(1, n, 6: Var proc_plus 

prop: Var function [proc.plus — > bool] 

lessp: function [proc_plus, proc.plus — bool] == (Xm t n : m < n) 
processors-induction: Lemma 

( V prop : prop(0) A ( V m : m < nrep A prop(m) 3 prop(m 4* 1)) 

3 ( V n : prop(n))) 

Proof 

Using noetlierian [proc.plus, lessp] 

reachability: Lemma a^0'»(36:a = 6+l) 

p_processor8-induction: Prove processors-induction {m *— h@P 2} from 
general-induction {p prop , d +- n, d? *- m}, 

reachability [a di®Pl} 

p-welLfounded: Prove well-founded {measure <— (A k — ► nat : k)} 
p- reachability: Prove reachability {6 ♦— if a = 0 then 0 else a - 1 end if} 

End 

sets: Module [T: Type] 

Exporting all 
Theory 
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set: Type is function[T — ► bool] 
z,y , 2 : Var T 
a, b: Var set 

★1 U *2: function [set, set — ► set] == ( A a, b : ( A x : a(x) V b(x))) 

★1 O *2: function[set, set — ► set] == ( A a, b : ( A x : a(x) A fc(x))) 

★1 \ *2: function [set, set — ► set] == ( A u, 6 : ( A x : a(x) A -*fc(x))) 
add: function[T, set — set] == (Ax,a:(Ay:x = yV a(y))) 
singleton: function[T — ► set] == ( A x : ( A y : y — x)) 

★ 1 C *2: function [set, set — ♦ bool] = ( A a, b : ( V z : a(z) D &{*))) 

*1 € *2: function [7\ set — ♦ bool] == (A x,fc : b(x)) 
empty: function[set — ♦ bool] = ( A 0 : (V x : -«a(x))) 

<j>: set == (Ax: false) 
fullset: set == ( A x : true) 

ex tension ali ty: Axiom = = 

End sets 

cardinality: Module [T: Type] 

Using sets[T] 

Exporting all 

Assuming 

x, y, z\ Var T 
N : Var nat 

/: Var function [T — ► nat] 

finite: Formula ( 3 N, f : ( V x, y : /(x) < N A (/(x) = f(y) D x — y))) 

Theory 

a, 6, c: Var set 

card: function [set —► nat] 

card.ax: Axiom card(a U 6) + card(a n 6) = card(a) + card(/>) 
card.subset: Axiom a C b 3 card (a) < card (6) 
card-empty: Axiom card(a) = 0 o- empty(a) 
empty-prop: Lemma card(a) >03(3x:x€a) 

card-prop: Lemma a C c Ab C c A2 + card(a) > card(c) A 2 * card(6) > card(c) 
3 card (a O 6) > 0 

Proof 

empty-prop_proof: Prove empty_prop {x x@p2) from card-empty, empty 

subset-union: Sublemma aCcAbCcDaUbCc 

subset_union_proof: Prove subset-union from 
★ 1 C *2 {z *— z®p3, b «— c} } 

*1 C *2 { z * — z®p3, a •*— fc, b <— c), 

*1 C *2 {a 4 — a U 6, b *— c] 

m, n, p: Var nat 

twice.prop: Sublemma 2 ♦ m > p A 2 * n > p 3 m + n > p 
twice.proof: Prove twice.prop 
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card-proof: Prove card .prop firom 

twice.prop {m «— card (a), n card (5), p <— card(c)}, 
card .ax, 
subset. union, 

card-subset {a <— a U 6, 6 «— c} 

End cardinality 
nat-inductions: Module 

Theory 

i,j: Var nat 

Var nat 

p: Var function [nat — ♦ bool] 
nat.complete: Axiom 

(Vm : (Vn 3 : (n 3 4 *i) 3p(*»a)) Dp( n O) ( v »2 ^(” 2 )) 
natjnduction: Axiom (p( 0) A ( V m : p(«i) 3 p(n t -1- 1))) 3 (Vn 2 : p(n 2 )) 
nat_induct-by_2: Axiom 

(p(0) Ap(l) A ( V ni : p(nj) 3 p(«i + 2))) 3 (Vn 2 : p(n 2 )) 

End nat .inductions 

noethcrian: Module [dom: Type, <: function[dom, dom —► bool]] 

Assuming 

measure: Var function [dom — ► nat] 
a, b: Var dom 

well-founded: Formula ( 3 measure : a < b 3 measure(a) < measure^)) 

Theory 

p t A, B\ Var fund ion [dom —*■ bool] 
d, J] , d 2 : Var doin 

general-induction: Axiom 

(Vdi : (Vd 2 : d 2 < d\ 3 p(d 2 )) 3 p(d,)) 3 (Vd :p{d)) 

End noetherian 
multiplication: Module 
Exporting all 
Theory 

V 2 ,r 2 : Var number 

★ 1 x *2: function [number, number — ► number] = ( A x, y : (x * y)) 

rnult-ldistrib: Lemma i x(y-fz) = r x^ + ixz 

in ui t .id istrib. minus: Lemma xx(y-z) = xxy-xxz 

mnlt-rident: Lemma ixl = i 

inult_lident: Lemma lxi = i 

distrib: Lemma (x + y)xz = xxz + yxz 

distrib_minus: Lemma (x — y)xz = xxz — yxz 
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mult.non.neg: Axiom ((x>0Ay>0)V(x<0Ay<0))<*xxy>0 

mult.poe: Axiom ((x > 0 A y > 0) V (x < 0 A y < 0)) * * V > » 

mult.com: Lemma x x y = y x x 

pos.product: Lemma x>0Ay>03x xy>0 

mult.leq: Lemma z> 0 Ai>yDxx 2 >j/X 2 

niultJeq.2: Lemma z>0Ai>y 3 ^xi>?xj/ 

mult. 10: Axiom 0 x x = 0 

mult.gt: Lemma z>0Ax>y3xxz>yxz 

Proof 

inult.gl.pr: Prove mult.gt from 

mult.pos {x ♦ — x — y, y ■*— z}, distrib.minus 

distrib.minus.pr; Prove distrib.minus from 
multJdistrib.minus {x <— z, y <- x, z y}, 
mult_com {x x — y, y <— 
miilt.com {y *— z) t 
mult.com {x ♦— y, y <— z} 

nuiltJeq.2.pr: Prove multJeq.2 from 

multJdistrib.minus {x «— z, y «— x, z «— y} f 
mult.non.neg {x «— z, y <— x ~ y} 

malt Jeq.pr: Prove mult Jeq from 

distrib.minus, mult.non.neg {x x — y, y z} 

mult.coin.pr: Prove mult.com from *lx*2,*lx*2{x<— y, jr *— x} 

pos.product.pr: Prove pos.product from mult.non.neg 

mult.rident.proof: Prove mult.rident from *1 x *2 {y 1} 

mullJident.proof: Prove mult.lident fVom *lx*2 {*♦-!, y «— *} 

distrib.proof: Prove distrib from 
*1 x *2 {x <— x 4- y, y «— z}, 

+1 x *2 jy z), 

*1 x *2 {x «- y, y *- z} 

mult Jdist rib. proof: Prove muUJdistrib from 

*1 x *2 {y ♦- y + z, x ♦— x), * l x *2 , *1 x*2 {y z) 

mult Jdistrib.ininus-proof: Prove miiUJdistrib.minus from 
x *2 {y — y - z, x «- x}, *1 x *2 , *1 x +2 {y z} 

End 

absmod: Module 
Using multiplication 
Exporting all 
Theory 
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z,U,*iZi>yi>zuZ2 t V2>zr> Var number 
| ★ 1 1 : Definition function [number — *■ number] = 

( A x : ( if x < 0 then - x else x end if)) 

abs.main: Lemma |x| < z D (x < z A — x < z) 

abs.leq.0: Lemma \x — y| < z D (x — y) < z 

abs.diff: Lemma \x — y| < z D ((x — y) < z A (y - x) < z) 

abs.leq: Lemma |x| < z D (x < z A -x < z) 

abs.bnd: Lemma 0OA0 <iAiOA 0 <yAj/<O|j~l/|<: 

abs.l.bnd: Lemma |x-y|<zDx<y + z 

abs.2_bnd: Lemma \x — y\<z^x>y~z 

abs_3_bnd: Lemma x<y + z/\x>y — z D \x - y\ < z 

abs.drift: Lemma \x — y| < z A |xj — x\ < z\ D \x\ — y] < z + zi 

abs.com: Lemma |x - y\ = |y - x| 

abs.drift.2: Lemma 

|x - y| < z A \x\ - x| < z\ A |yi - y| < Z 2 D |xi - yi | < z + z\ -f *2 
abs.geq: Lemma x > y A y > 0 D |x[ > |y| 
abs.geO: Lemma x > 0 3 |x| = x 
abs.pl us: Lemma |x -h y| < \x\ -f |y| 
ab$_difF.3: Lemma x — y < z A y — x < z 0 \x — y\ < z 
abs.eq: Lemma |x — y\ = |y - x| 

Proof 

abs.plus.pr: Prove abs.plus from | * 1| {x 4— j -f y} t ] a- 1| , | * 1| (x y} 

abs.diff_3-.pr: Prove abs_diff_3 from | * 1| {x x — y} 

abs_geO_proof: Prove abs.geO from | * 1| 

abs_geq_proof: Prove abs.geq from | * l| , | + l| {x *- y] 

abs.drift.2.proof: Prove abs_drift_2 from 
abs.drift, 

abs.drift {x +- y, y — yi, z «- zu z t *- z + z j }, 
abs.com {x * — yi } 

abs.com.proof: Prove abs.com from |*1| {x<— (x-y)}, | * 1| {x <— (y — x)} 
abs.d rift.proof: Prove abs.drift from 


abs.l.bnd, 




abs.l.bnd {x 

— Xi, 

y — x, z — 


abs.2.bnd, 




abs.2.bnd {x 

<- X U 

Jf^x, z +- 

-1}, 

abs.3.bnd {x 


z 4- z + XI } 



abs_3.bnd_proof: Prove abs.3_bnd from | * l| {x 4- (x - j/)} 
abs-inain.proof: Prove abs.main from |* 1| 
absJeq.O.proof: Prove absJeq.O from | * l| {x <— x - y) 
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abs.diff.proof: Prove abs.diff from | * 1| {x «- (x - y)} 
abs.leq.proof: Prove abs .leq from | * 1 1 
abs.bnd.proof: Prove abs.bnd from | * l| {x «- (x - y)} 
abs.l.bnd.proof: Prove abs.l.bnd from | * 1| {x — (x - j/)] 
ab8_2_bnd.proof: Prove abs_2.bnd from |*l| {x <— (x — y)} 

End absmod 
rcp.defs.tcc: Module 
Using rcp.defs 
Exporting all with rcp.defs 
Theory 

p: Var naturalnumber 
m: Var proc.pl us 
a: Var proc.plus 

prop: Var function [proc.plus — *■ boolean] 
d\ : Var proc.pl us 

6; Var proc.plus (* Existence TCC generated for processors *) 
processors.TCCl : Formula ( 3 p : (p > 0) A (p < nrep)) 

proc.plus.TCCl: Formula ( 3 p : (p > 0) A (p < nrep)) 

processors.induction.TCCl: Formula ((0 > 0) A (0 < nrep)) 

processors.induction.TCC2: Formula 

(m < nrep A prop(m)) A (prop(O)) 3 ((m + 1 > 0) A (m + 1 < nrep)) 

p.reachability.TCCl; Formula 

( if a ss 0 then 0 else a — 1 end If > 0) 

A (( if a = 0 then 0 else a - 1 end if > 0) 

A ( if a =s 0 then 0 else a - 1 end if < nrep)) 

Proof 

processors.TCCl.PROOF: Prove processors.TCCl 
proc.plus.TCCl -PROOF: Prove proc.plus.TCCl 

processors.induction.TCCl.PROOF : Prove processors.induction.TCCl 
processors_induction_TCC2_PROOF: Prove processor8.induction.TCC2 
p.reachability.TCCl. PROOF: Prove p.reachability.TCCl 
End rcp.defs.tcc 
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Appendix B 

LaTeX-printed Supplementary Specification Listings 


rcp-defs: Module 

(* This rcp.defs Module differs slightly froM the original. Several 
definitions have been moved to new modules; the originals have 
been commented out. *) 

Exporting all 
Theory 

p: Var nat 

inputs: Type (* type of external sensor input *) 

outputs: Type (* actuator output type *) 

nrep: nat (* number of replicated processors *) 

recovery-period: nat (* number of healthy frames required to recover 

from transient fault plus one *) 
recovery_period_ax: Axiom recovery-period > 2 

processors_exist-ax: Axiom nrep > 0 

processors: Type from nat with ( A p : (p > 0) A (p < nrep)) 
phases: Type = (compute, broadcast, vote, sync) 
ph: Var phases 

next-phase: function [phases — ► phases] = 

( A ph : if ph = compute 
then broadcast 

elsif ph = broadcast then vote elsif ph = vote then sync else compute 
end if) 

prev.phase: function [phases — *■ phases] = 

( A ph : if ph = compute 
then sync 

elsif ph — broadcast 
then compute 

elsif ph = vote then broadcast else vote 
end if) 

proc.pl ns: Type from nat with ( A p : (p > 0) A (p < nrep)) 

Var proc-plus 

prop: Var function [proc.pl us — ► bool] 

lcssp: function [proc-pl us, proc_plus — * bool] == ( A m, n : m < ») 
processors.induction: Lemma 

(V prop : prop(O) A ( V m : m < nrep A prop(m) 3 prop(m -f 1)) 

3 ( V n : prop(n))) 

Proof 

Using noetherian [proc.plus , lessp] 

reachability: Lemma a/0o (3fc:a = fc + l) 

p_processors_induction: Prove processors-induction {m «— b@P'2) from 
general-induction [p <— prop, d «— n, d? *— m), 
reachability {u <— 

p_ well-founded: Prove welLfoumled {measure «— ( A k — *■ nal : k)} 

P-reach ability: Prove reachability {b if a = 0 then 0 else a — 1 end if} 
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End 

task-model: Module 


(* This module introduces an interpretation for a basic task-oriented 
style of computation state. It is common to both the continuous 
voting and cyclic voting interpretations. *) 

Using rcp.defs, sets [processors] , cardin al i ty [processors] , nat_i nd uctious 
Exporting all with rcp.defs, sets[processors], cardinality [processors] 

Theory 

р, i, j: Var processors 
k, /, q : Var nat 

u: Var inputs 

A: Var set[processors] (* Basic definitions for schedules *) 
maj.condition : function [set[processors] — *• bool] = 

( A A : 2 * card(yt) > card (fullset [processors])) 
schedule-length: nat (* Humber of frames in schedule cycle *) 
schedule-length-ax: Axiom schedule-length > 0 

control-state: Type from nat with ( A k : k < schedule-length) 

A', L\ Var control-state 

mod-plus: function [control-stale, control-state — ► control-state] = 

( X K } L — * control-state : 

if K -f L > schedule-length 
then K + L — schedule-length 
else K + i 
end if) 

inod_ininus: function [control-state, control-state — * control-state] = 

( A K t L — control-state : 

if K > L then K — L else schedule-length - L -f I\ end if) 
num.cells: nat 

num-cells-ax: Axiom num.cells > 0 

cell: Type from nat with ( A k : k < num-cells) 
cell-state: Type 

cell-array: Type = array [cell] of cell-state 

с, d,e: Var cell 
fl : Var nat 

(7, D: Var cell-array 


(♦ Task schedule concepts. Each cell occupies a unique place in the 
schedule, being computed only once per schedule cycle. *) 
cell-frame: function [cell — control-state] (* scheduled frame of cell *) 
cell-subframe: function [cell — nat] (* scheduled subframe of cell *) 
sched-cell: fu nction [control-state, nat — cell] (* cell of frame, subframe ♦) 
nuni-subframes: function[control_state — * nat] (* subframes for this frame *) 


(* Well-formedness axioms constraining these functions *) 

celLframe-^x: Axiom r = sdiccLcellfA', k) 3 ccll-framc(c) = K 
celLsubframe-ax: Axiom c = sched.cell (K,k) D celLsubframefc) = k 
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sched-celLax: Axiom 

K = celLframe(c) A k — celLsubframe(c) 3 sche<Lrell( A , k) = r 
nuni_subframes_ax: Axiom 

K = cell, f ram e(c) 3 celLsubframe(c) < nnni_subframes(/\') 

(* Processor state definition *) 

Palate: Type = Record control : control-state, 
cells : cell.array 
cud record 

nulLcelLarray: celLarray (* default value *) 

iniliaLproc-state: Pstate (* assumes each processor begins identically *) 
MB: Type is Pstate 

MBvec: Type = array [processors] of MB 

MB matrix: Type = array [processors] of MBvec 

w\ Var MBvec 

h: Var MBmatrix 

us, ps, A , Y : Var Pstate 

cell. array-equal: Axiom ( Vc : C(c) = 0(c)) 3 C = D 

Pstale-equal: Axiom (.Y.control = Yxonlrol A X. cells = YxelLs) 3 A" = Y 

(* Interpretations for task-related functions *) 
succ: function [controLstate — ► controLstate] = 

( A A' — ► controLstate : 

if K + 1 < schedule-length then K + 1 else 0 end if) 

/*: function [Pstate — ► controLstate] == ( A ps : ps. control) 

/,: function[Pslate, cell — cell-state] == ( A ps,c : ps.cells(c)) 

(* Functions modeling task execution *) 


exec_task: function [inputs, controLstate, cell-array, nat — + celLstate] 


exec-measure: function [in puts, control-state, celLarray, nat — ► nat] 

( A u, A', <7, k : k) 

exec: Recursive function [inputs, controLstate, celLarray, nat 

— - celLarray] = 

(A u,A\C,* : 
if k = 0 
then C 

else exec(u, K t C,k — 1) 
with [(sched_cell(A\ k - 1)) : = 

exec_task(ti, K,exe.c(u, f\\C,k — 1 ),k — l)] 
end if) 

by exec.measure 

f c : functionfinputs, Pstate — ► Pstate] = 

( A u, ps : ps with [(control) := succ(ps. control), 

(ceils) := exec(u, ps. control, 

ps. cells, num_suhfraines(ps. control))]) 


f a '. function[Pstate — ► outputs] (* actuator output ♦ ) 

(♦ Axioms to be satisfied by the generic application *) 
succ.ax: Formula A(/V(ti*ps)) = su<x(/ fc (ps)) 
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components.equal: Formula /*( A) — fk(Y) A(Vr : /*( A,r) — /iO>c)) D ^ 
(* Support leMiaa *) 


succ.Ie.plus: Lemma succ(A') < A' + 1 

rnod.minus.zero: Lemma mod.minus(A', L) = 0 ^ K = L 

mod.minus.succ: Lemma mod_minus(succ( K) t L) — succ(rnod_minus(A', L)) 

mod.minus.plus: Lemma succ( K) ^ /O mod_niinus(succ( A ), L) = mod_nnitus(A , L) 4- 1 

exec.element: Lemma 

exec(u, A', <7, nuni -subframes ( A'))(c) 

= if celLfrarne(c) — K 

then exec_task(ii, A',exec(n, A' t C f celLBubfranie(c)),«!lI-«iibfraiiie(c)) 
else C(c) 
end if 


Proof 

p.succ.ax: Frove succ.ax from f c 

p.components.equal: Prove components.equal {c <— from 

cell-array-equal {C +- X.cells, D +- Vxells}, Pstate.cqual 

p.succ.le.plus: Prove succ.le.plus from succ 

p_mod.minas.zero: Prove mod.minus.zero from mod.minus 

p.mod.minus.succ: Prove mod.minus_succ from 

mod.minus {A* * — succ(A^)}, mod .minus, succ {A < — uiod_iniiius(A , />)}, succ 

p_mod.minus.plus: Prove mod_rninus_plus from 
mod.minus {K <— succ(A )}, mod.ininus, succ 

exe.prop: function [inputs, control.state, cell-array, cell, nat 
— ► function [nat — *■ bool]] = 

( A u, A', <7,c, k : 

( A q : cell.su bframe(c) = k A q < num.subframes( A') 

D exec(n, A', C, q)(c) 

— \fk<q/K cell_frame(<;) = K 

then exec_task(u, A Jr ,excc(u, K , C t k ), fc) 
else C(c) 
end if)) 

exe.base: Lemma exe_prop(u, A\ C , c, fc)(0) 

exe.ind.l: Lemma exe.prop(u, K } C y c, fc)(g) A celLsubframe(c) = q 
D exe_prop(u, A",C,c,fc)(? -f 1) 

exe_ind_2: Lemma exe_prop(u, I\ t <7, c, fc)(?) A celLsubframe(e) ^ ? 

D exc.prop(u, K y <7, c, fc)(g + 1 ) 

p.exe.base: Prove exe.base from exe.prop {g «- 0}, exec {fc 0) 

p_exe_ind_l: Prove exe.ind.l from 
exe.prop {</ — </}, 
exe.prop {q «— q + 1}. 
exec {fc *— q + 1 }, 
schcd.celLax {fc «— </}, 
cell.frame.ax {fc «— </}, 
n u m.su bfr am es. ax 
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p_exeJnd.2: Prove exe-ind-2 from 
exe.prop {? — ?), 
exe.prop {?*-? + 1}, 
exec {k «— q + 1 }, 
ceILframe_ax { k *— q], 
celLsubframe-ax {k «— g}, 
nmnjsubframes_ax 

P-exec-element: Prove exec.elcinent from 
nat-induclion 

{p ♦— exe-prop(tt, K t C, c, cell-subframe (e)), 
ri 2 <— nuin-suhfranies(A')}, 

exe.prop <— mun_subframes(A'), k <— celLsubfraine(r)}, 
exe.base {k c:elLsubframe(c)}, 
exeJnd-l {q «— njflpl, k *— celLsubframe(c)}, 
exe_ind-2 {g «— «i@pl, k cdLsubframe(c)}, 
nu m_sub frames, ax 

End 

cont. voting: Module 

(♦ Following is the interpretation for the continuous voting scheme. *) 

Using task_modd, nat-inductions 

Exporting all with task-model 
Theory 

us, ps, A", Y: Var Pstate 

р, i t j: Var processors 
k t l y q: Var nat 

u: Var inputs 
w: Var MBvec 
h: Var MBmatrix 
A: Var set [processors] 

с, d } e: Var cell 
cs: Var cell-state 

K: Var control-state 

//: Var nat (* Majority functions ♦) 

k-inaj: function [MBvec — ► control-state] 

k-maj.ax: Axiom ( 3 A : 

maj_condition(yl) A (Vp : p £ /4 3 ic(p).conlrol = A')) 

3 k_maj(ic) = K 

t.maj: function [MBvec, cell — ► celLstate] 
t_maj-ax: Axiom (3 >4 : 

maj-Condition(A) A (Vp :p £ 4 3 ((u>(p)).cellsc) = cs)) 

3 t_maj(u;, c) — cs 

celLmeasure: fu net ion [MBvec, nat — ► nat] == (A w t k : k) 
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celLmaj: Recursive function [M Bvec, n at — ► celLarray] = 
( A u>, Jb : if Jt = 0 V A > num-cells 
then nulLcelLarray 
else cell_maj(u;, k - 1) 
with [(Jfc - 1) := t-maj(tn, k - 1)] 
end if) by cell-measure 


(* Interpretations for voting-related functions ♦) 


function [Pstate — ► MB] == ( A ps : ps) 


f v : function [Pstate, MBvec — ♦ Pstate] = 

( A ps, w : ps with [(control) := k_maj(w), (cells) := 
cell.maj(w, num .cells)]) 


rec: function [cell, controLstate, nat — bool] == (Ac, A’, II : If > 2) 

dep: function [cell, cell, controLstate — ► bool] == ( A c,d, A : false) 
recovery-period- value: Axiom recovery-period = 3 
(* Definitions derived fro* uninterpreted functions *) 

dep-agree: function [cell, controLstate, Pstate, Pstate — ► bool] = 

( A c, A, A", Y : (Vd : dep(c,d, A') 3 f t (X,d) = f t (Y> d))) 

w .condition : function [set[processors], MBvec, Pstate — ► bool] = 

( A A, tn.ps : (Vp : p € A 3 tu(p) = /,(ps))) 

(* Ax ions to be satisfied by the generic application *) 
full-recovery: Formula H > recovery-period 3 rec(c, K, II) 
initial- recovery: Formula rec(c, h\If) 3 fl > 2 

dep-recovery: Formula rec(c, succ(A'), If + 1) A dep(c, d , K) 3 rec (d, K , II) 
control- recovered: Formula 

inaj_condition(/l) A ( Vp : j) € A 3 w(p) = /j(p*0) 3 A(/«( V, tn)) = /*(ps) 

cell- recovered; Formula 
maj_condition (y\) 

A ( V p : p € A 3 w(p) = /•(/*(«. P s ))) 

A /*(AT) = A' A /jt(ps) = A' A dep_agree(c, A\A\ps) 

3 /i(/4/c(tt,.V) f w),c) = / t (/ c (u,ps),c) 

vote_maj: Formula 

maj-condition(A) A ( Vp : p € .4 3 tu(p) = f*(P s )) 3 u> ) — P s 

(♦ Support leuae ♦) 

celLmaj-element: Lemma celLmaj(tu, num-cclls)(c) = t-iuaj(u», c) 
f-v-componcnts: Lemma /fc(/w(ps,«0) ~ k_maj(u>) A /t(/«(ps, in), c) = l-inaj(u»,c) 
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Proof 


p.fulLrecovery: Prove fulLrecovery from recovery-period- value 
p.initiaL recovery: Prove initial- recovery 
p-dep_recovcry: Prove dep.rccovery 

p_control_recovered: Prove control- recovered {p p®pi} fVoin 
k-inaj_ax {A' «— ps. control}, f v {ps «— Y , w +- tu} 

p .cell-recovered: Prove cell-recovered [p *— p@pl} from 
t.maj-ax {cs <— ((/ c («, ps)). cellar)}, 
fc {ps «- -V), 
fo 

fv {ps *- /c(u, A'), w *- «;}, 
cell-maj-element 

P-vote-niaj: Prove votejmaj {p «— p®p 4 } from 
components_equal {A" <— / v (ps, u>), Y «— ps), 
k.maj.ax {A" <— ps.control}, 
t_maj-ax {cs <— f>s(.cellsc@pl), c <— c®pl}, 
w -condi lion, 

w .condition {p p@p2}, 
w_condilion {p ♦— p@p 3 }, 
f_v -components {c e@pl) 

cme-prop: fnnction[MBvec, cell — ► function [nat — ♦ bool]] = 

( A w,r : ( A 9 : 

cell_maj(ut,</)(c) 

= if c < 9 A7 < num.cells 
then t.maj(u>,c) 
else nulLcelLarray(c) 
end if)) 

cme.base: Lemma cme.prop(u>, c)(0) 

cme_ind_l: Lemma cme.prop(tt», c)(q) A c — q D cme_prop(tt>,c)(<7 + 1 ) 

cme.ind. 2 : Lemma cme.prop (w,c)(q) Ac / 9 D cme_prop(u>,c)(<7 + 1 ) 

p.cme.base: Prove cme.base from cme.prop {q «— 0}, celLmaj {k *— 0} 

p_cmeJnd_l: Prove cmeJnd -1 from 

cme.prop {q 9}, cme.prop {7 q + 1}, celLmaj {k q -f 1 } 

p.cmeJnd.2: Prove cmeJnd_2 from 

cme.prop {q «— g}, cme.prop {<7 *- <7 + 1 }, celLmaj {k «- q -f 1 } 

P-celLmaj.element: Prove cell_maj_eleinent from 

nat .induction {p <— cme_prop(u>, c), n 2 <— num.cclls}, 

cme.prop {q <— num.cells}, 

cme.base, 

cmeJnd.l {g ♦— ni@pl}, 
cmeJnd_2 {q ♦— nj@pl} 

p.f.v .components: Prove f_v.com ponents from f v , celLniaj.eleincnt 

End 
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cyclic-voting: Module 

(* Following is the interpretation for the cyclic voting scheme. ♦) 

Using task-model, nat-inductions 
Exporting all with task-model 

Theory 

ns, ps, A', Y : Var Pstate 
p f i,j: Var processors 
kj } q: Var nat 
«: Var inputs 
ti>; Var MBvec 
h: Var MBmatrix 
A: Var set [processors] 
c, d, e: Var cell 
cs: Var cell-state 
K , L'- Var control-state 
//: Var nat 
C, 1): Var cell-array 

celLfn: Typo is function [cell — ► cell-state] 
cfn: Var cell-fn (* Majority functions *) 
k-ruaj: function [MBvec -► control-state] 

k-maj-ax: Axiom ( 3 A : 

niaj-coiulition(/4) A (V p : p € A D w(p). control = ft')) 

3 k.maj(w) = K 

t.maj: function [MBvec, cell celLstate] 
t-maj-ax: Axiom (3 A : 

maj-condition(A) A ( V p : p € A 3 ((tn(p)). celiac) = cs)) 

3 t_maj(tt>, c) = cs 

cell-measure: function [cell-fn, control-state, cell-array, nat * nat] 

( Acfn, A',C, fc : k) 

cell.apply: Recursive function [cell.fn, control-state, cell-array , nat 

— ► cell, array] = 

( A cfn, K y C, k : 

if k = 0 V k > num.cells 
then C 

elsif K = succ(celLframe(lr - 1)) 
then celLapply(cfn, K,C, k - 1) 
with [(k — 1) := cfn (A: — 1)] 
else cell.apply (cfn, K , £7, k — 1) 
end if) 
by cell. measure 

(* Interpretations for voting-related functions *) 


function [Pstate 
( A ps : ps with 


- MB] = 

[(control) := ps. control, (cells) := 
cell.apply (( A c : ps. cells (c)), 
ps. control, 
nulLcelLarray, 
limn -cells)]) 
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fv ' 


function [Pstate, MBvec — ► Pstate] = 

( A ps, xv : ps with [(control) := k_maj(u;), 

(cells) := celLappIy(( A c : t_maj(w, c)), 
pn. control, 
ps. cells, 


nuni-cella)]) 


rcc: function [cell, control-state, nat — + bool] = 

(Ac, KJf : ll 

> 1 + ( if K = celLframo(c) 
then schedule-length 
else mod-ininus(A",celLframe(r)) 
end if)) 


dep: function [cell, cell, control-state — ♦ bool] = 

( A c,d, K : celLframe(c) / K Ac = d) 

recovery-pcriod-value: Axiom recovery-period = sclicd ule_ length +2 

(* Definitions derived from uninterpreted functions *) 
dep.agree: function [cell, control-state, Pstate, Pstate — ♦ bool] = 

( A c, A\ X, Y : ( V d : dep(c, cf, K) D /,( A', d) = f t (Y, d))) 


w -condition : function [set [processors], MBvec, Pstate — ► bool] = 
( A A, w, ps : ( V p : p € A O to(p) = /.(ps))) 

(* Axioms to be satisfied by the generic application *) 


full- recovery: Formula 7/ > recovery-period 0 rec(c, K , //) 
initial-recovery: Formula rec(c, K\ H) 3 11 >2 

dep.rccovery: Formula rec(c, succ(/\'), II + 1) A dep(c,</, K) D rec (d, 7v , //) 
control, recove red: Formula 

maj.condition(yi) A (Vp : p e A D tt>(/>) = /,(ps)) D /*{/ w (y, in)) = /jt(ps) 

cell, recovered: Formula 
maj^condition (A) 

A ( Vp : p e 3 w(p) = /.(/c(u, ps))) 

A /*(.Y) ss 7\' A /*(ps) = A' A dep-agree(c, K y X, ps) 

3 /l(/»(/c(«, .V),tu),c) = /i(/c(u,ps),c) 

vote-maj: Formula 

maj-condition(/i) A(Vp:p€ /4 3 u>(p) = /,( ps)) 3 /„(ps, u») = ps 
O Support lemmas *) 

cell, apply .dement: Lemma 

cell_apply(cfn, K , C, num-cells)(c) 

— If A" = succ(ccil_frame(c)) then cfn(c) else C{c) end if 
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Ls.components: Lemma 

K = ps. control D /*(/*( ps)) = X 

A/t(/ a (ps),c) 

= if succ(cell_frame(c)) = K 
then ps.cells(c) 
else nulLcclLarray(r) 
end if 

Lv.components: Lemma 
/*(/.( P»>txO) = k-maj(tp) 

A ACMp*. c ) 

= if succ(celLfranie(c)) = ps. control 
then t_inaj(to,c) 
else ps.ce)ls(c) 
end if 


Lc-uncomputed-cclls: Lemma 

celLfraine(c) ^ A'.control D / c («. A"). cells (c) = X xe\\s(c) 


Proof 


p.fulL recovery: Prove full-recovery from 
rec, 

recovery -period - val u e , 
control-state Jnvariant 

{eontroLstate_var «— mod_ininus( A , cell_fraine(c@pl ))} 


pjnitial- recovery: Prove initial- recovery from 
rec, 

schedule-length-ax, 

mod-minus_zero {L *- celLfranie(c@pl)}, 

nat -invariant {nat_var «— ■ mod_minus(A ,cell_fraine(c@pl))} 


p_dep_recovery: Prove dep_recovery from 
rec {K «— succ(A'), // <— H -f l}i 
dep, 

rec {c 4— d), 

controLstateJn variant {control_state-var 
mod-minus-plus {L *— cell.frame(c)} 


mod. minus (A\ cell-frainc(c))}, 


p-control- recovered: Prove control-recovered {p «— p®pl} from 
k-maj-ax {A' <— ps. control}, /,. {ps «— Y, w w} t f a 


p_cell-recovered: Prove cell-recovered {p p@pl} from 
t_maj_ax (cs <— ((/«(/ c («, ps))).cellsc)}, 
dep-agree [Y ps, d «— c}, 
dep {d <— c}, 

f_s-Components {ps «— / c («, P s )> A" «— (/ c (u, A')). control), 
Lc-uncomputed-cells {X «— ps}, 

Lc_uncomputed-cells, 
fc {ps — X), 

fc , 

Lv .components {ps / C («,A')} 


i 
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p.vote.maj: Prove vote_maj {p «— p@p4} from 
componenls.equal {A r ♦— f v { ps,u»), Y +- ps}, 
k.maj.ax {K «— ps. control}, 
t_maj.ax {cs <— ps(.cellsc@pl), c «— c%> 1), 
w_condition, 

w .condition {p *— p@p2}, 
w .condition {p <— p@p3}, 

celLapply-cIcmcnl 

{cfn «— ( A r : ps.cells(c)), 
c c@pl, 

K «— ps. control, 

C <— nulLcelLarray}, 
f.v .components {c «— c@pl] 

cae.prop: function [cell.fn, control-stale, cdL array, cc It 
— * function [nat —► bool]] = 

( A cfn, K % <7, c : 

(A q : cell_apply(cfn, K,C } q)(c) 

= if c < q A q < nmn.cells A K = succ(cell_frame(c)) 
then cfn(c) 
else (7(c) 
end if)) 

cae.base: Lemma cac_prop(cfn, A',C,c)(0) 

cae.ind.l : Lemma cae.prop(cfn, A', (7, c)(?) A c = q 
D cae_prop(cfn, K t C t c)(q - f 1) 

cae.ind.2: Lemma cae_prop(cfn, K,C, c)(q) Ac ^ q 
Z> cae_prop(cfn, A\ (7, c)(q 4- 1) 

p.cae.base: Prove cae.base from cae.prop {q *— 0}, celLapply {A: ♦- 0} 

p.caeJnd.l: Provo caeind.l from 

cae.prop {q «— ?}, cae.prop {q +— q + 1}, celLapply {k *— q + 1} 

p_caeind_2: Prove caejnd_2 from 

cae.prop {?<-?}, cae.prop {q «- q 4- l), celLapply {k <- q + 1} 

p.celLapply.element: Prove cell.apply .element from 

natJnduction {p cae_prop(cfn, A\(7,c), u 2 «— num.cells}, 

cae.prop {q *— num.cells}, 

cae.base, 

caeJnd.l {g <— hi®/> 1}, 
caeJnd_2 {g ♦— m©pl} 

p.Ls.components: Prove f_s.com ponents from 

/. , 

cell.apply.element 

{cfn (Ar : ps.cells(c)), 

K «— ps. control, 

C «— nulLcelLarray} 

p_Lv.com pone tits: Prove f_v.com ponents from 

fv , 

celLapply.cleinent 

(cfn ^ (Ac: t_maj(tn, e)), 

K ps. control, 

<7 «— ps. cells} 
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pXc-uncomputed-cdls: Prove f_c_u ncom pu totLcdls from 
fc {ps -V}, exee.element {C «— A'.cells, K «— X. control] 

End 



Appendix C 

Results of Proof Chain Analysis 


The following pages were obtained from Elidm using the proof-chain analyzer command (M-x apes) applied to the 
module top. 

Terse proof chains for nodule top 

Use of the formula 

RS.to.US . f rame.commutes 
requires the following TCCs to be proven 
RS.to.US.tcc . reachable.in.n.TCCl 
RS.to.US.tcc . reachable_in_n_TCC2 

Formula RS_to.US.tcc .reachable_in_n_TCC2 is a termination TCC for 
DA.to.DS . reachable. in.n 
Proof of 

RS.to.US.tcc . reachable_in_n_TCC2 
must not use 

DA.to.DS . reachable.in.n 

Use of the formula 

rcp.def s . recovery .period.ax 
requires the following TCCs to be proven 
rcp.def s. tcc . processors. TCC1 
rcp.def s_tcc .proc. plus. TCC 1 
rcp.def s. tcc . processors.induct ion.TCC 1 
rcp.def s_tcc.processors_induction_TCC2 
rcp.def s. tcc .p.reachability.TCCl 

Use of the formula 

cardinality [rcp.def s. processors] .card.empty 
requires the following assumptions to be discharged 
cardinality [rcp.def s .processors] . f ini te 

Use of the formula 

DS.to.RS . frame. commutes 
requires the following TCCs to be proven 
DS.to_RS_tcc . ss.update.TCCl 
DS.to_RS.tcc . ss_update_TCC2 
DS.to_RS.tcc . ss.update.TCC3 
DS_to_RS_tcc.HBmatrix_cons.TCCl 

Formula DS.to.RS.tcc.ss_update.TCC3 is a termination TCC 
for DA.to.DS. ss.update 
Proof of 

DS.to_RS.tcc . ss_update_TCC3 
must not use 

DA.to.DS . bs. update 

Formula DS.to_RS.tcc.MBmatrix.cons.TCCl is a termination TCC for 
DA.to.DS .MBmatrix.cons 
Proof of 

DS.to.RS_tcc.HBmatrix_cons.TCCl 
must not use 

DA.to.DS .MBmatrix.cons 
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Use of the formula 

noetherian [rcp.def s . proc.plus , rcp.def s . lesspl . general. induct ion 
requires the following assumptions to be discharged 

noetherian [rcp.de fa. proc.plus, rcp.def s .lessp] .well-founded 

Use of the formula 

DS.support. proof . si 13_prop 
requires the following TCCs to be proven 
DS.support.proof _tcc .p.sll3_base_TCCl 
DS.support.proof .tcc .p.sl 13_ind_TCC 1 
DS.support.proof .tcc .p_support.13.TCCl 

Use of the formula 

DS.map.proof .mil .prop 
requires the following TCCs to be proven 
DS.map.proof _tcc.p_mll_base_TCCl 
DS.map.proof _tcc .p.mll.ind.TCCl 
DS.map.proof _tcc .p.map.l.TCCl 

Use of the formula 

DA.to.DS . phase -commute 8 
requires the following TCCs to be proven 
DA.to.DS.tcc .ss.update.TCCl 
DA.to.DS.tcc .ss.update.TCC2 
DA.to.DS.tcc . ss.update.TCC3 
DA.to.DS.tcc .HBmatrix.cons.TCCl 
DA.to.DS.tcc .reachable.in_n.TCCl 
DA.to.DS.tcc . reachable.in_n.TCC2 

Formula DA.to.DS.tcc. ss_update_TCC3 is a termination TCC 
for DA.to.DS. ss.update 
Proof of 

DA.to.DS.tcc . ss.update.TCC3 
must not use 

DA.to.DS . ss.update 

Formula DA.to.DS.tcc. HBmatrix.cons.TCCl is a termination TCC for 
DA.to.DS .HBmatrix.cons 
Proof of 

DA.to.DS.tcc .HBmatrix.cons.TCCl 
must not use 

DA.to.DS . HBmatrix.cons 

Formula DA.to.DS.tcc. reachable.in_n.TCC2 is a termination TCC for 
DA.to.DS . reachable. in.n 
Proof of 

DA.to.DS.tcc . reachable. in.n_TCC2 
must not use 

DA.to.DS . reach able. in.n 

Use of the formula 

DA.map.proof .mil .prop 
requires the following TCCs to be proven 
DA.map.proof .tcc .p.mll.base.TCCl 
DA.map.proof .tcc . p.mll.ind.TCC 1 
DA.map.proof _tcc .p.map.l _TCC 1 
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Use of the formula 

DA.br oadc as t.prf .rtp4 
requires the following TCCs to be proven 
DA.broadcast.prf _tcc .p_br8_TCCl 

Use of the foraula 
clkaod. in.R.interval 
requires the following TCCs to be proven 
clkaod.tcc. half _TCC1 
clkaod.tcc . Rho.TCCl 
clkaod.tcc . Corr.TCCl 
clkaod.tcc . nua.good_clocks.TCC 1 
clkaod.tcc .nua.good_clocks.TCC2 
clkaod.tcc . nua_good_clocks_TCC3 
clkaod.tcc. nun_good.clockB.TCC4 
clknod_tcc.nun.good. cl ocks.TCC5 
clkaod_tcc.C6.TCCl 

Foraula clkaod.tcc . Corr.TCCl is a termination TCC for clkmod.Corr 
Proof of 

clkaod.tcc . Corr.TCC 1 
must not use 
clkmod.Corr 

Foraula clkaod.tcc ,nua.good.clocks.TCC4 is a termination TCC for 
clkaod . nua.good.c locks 
Proof of 

clkaod.tcc . nua_good_clocks_TCC4 
must not use 

clkaod . nua.good.c locks 

Formula clkaod.tcc .nua.good.clocks.TCC5 is a termination TCC for 
clkaod . nua.good.c locks 
Proof of 

clkaod.tcc . nua.good. clocks. TCCS 
must not use 

clkaod, num.good.clocks 

Use of the foraula 

DA. invariants. state. invariant 
requires the following TCCs to be proven 
DA.invariants.tcc .Corr.lea.TCCl 

Use of the foraula 

DA.support .proof . si IS.prop 
requires the following TCCs to be proven 
DA.support.proof _tcc ,p_sl 13_base.TCCl 
DA_support.proof.tcc.p_sll3.ind.TCCl 
DA.support.proof. tcc.p_support_13.TCCl 
DA.support.proof _tcc . p_sl 15_ind_TCC 1 

SUMMARY 

The proof chain is complete 

The axioms and assumptions at the base are: 

DA . all.durat ions 
DA . broadcast. duration 
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DA .broadcast _dur at ion2 

DA . pos.durat ions 

RS.ma jority . ma j.ax 

cardinality [EXPR] .card_empty 

clkmod.CO 

clkaod.Cl 

clkmod. C2 

clkmod . Theorem. 1 

clkmod . Theore*_2 

clkmod . adj.always.pos 

generic.FT. cell.recovered 

generic.FT. component s.equal 

generic, FT. control.recovered 

generic, FT . dep.recovery 

generic.FT. full.recovery 

generic.FT. initial.recovery 

generic, FT . succ.ax 

generic.FT . vote.ma j 

multiplication. aul t .non.neg 

nat, inductions .nat, induct ion 

noether ian[EXPR, EXPR] .general_induction 

rcp,def s . processors,exist,ax 

rcp.def s . recovery, per iod,ax 

sets [EXPR] .extensionality 

Total: 26 

The definitions and type-constraints are: 
DA . H.da 

DA . N.da.broadcast 

DA.H.da.compute 

DA.H.da.sync 

DA. H.da, vote 

DA.broadcast.received 

DA . clock, advanced 

DA.da,rt 

DA . enough.hardware 

DA . initial, da 

DA. maj. working 

DA.working.proc 

DA. working.set 

DA,invariants . cum.delta.val 

DA, invariants . lclock,eq 

DA.invariants . lclock,val 

DA, invariants . nf _clks 

DA, invar iants . state_invariant 

DA.invariants. state, invariant_to,n 

DA. lemmas . hide 1 

DA.map.pr oof .mil, prop 

DA.map.proof ,ml2_prop 

DA.map.proof .ml4_prop 

DA.support.proof . si tS.prop 

DA.to.DS . DAmap 

DA.to.DS . reachable 

DA.to.DS . reachable.in.n 

DA.to.DS . ss.update 

DS.M.ds 

DS . N.ds .broadcast 
DS . H.ds.compute 


DS.M.ds.sync 

DS.M.ds.vote 

DS . allowable. faults 

DS .broadcast .received 

DS.frame_II.ds 

DS.initial.ds 

DS.maj .working 

DS . working_proc 

DS. working.Bet 

DS.lemmae .half _f rame.H.ds 

DS.lemmas .quarter.f rame.H.ds 

DS.map.proof .all .prop 

DS.map.proof . «12_prop 

DS.support.proof . al 1 3_prop 

DS.to.RS.DSmap 

DS_to_RS.MBmatrix.cons 

DS.to.RS . good.values.sent 

DS.to.RS . is.new.proc.state 

DS.to.RS . ss.update 

DS.to.RS . voted.f inal.state 

RS.M.rB 

RS . allowable.f aulte 
RS . good.values.sent 
RS . initial.rs 
RS.maj. working 
RS . voted.f inal.state 
RS . working_proc 
RS.working.Bet 

RS.invariants . state. invariant 

RS. invariants . Btate.invariant.to_n 

RS.lemmas . cell.recovery 

RS.lemmas .control.recovery 

RS.lemmas .state, recovery 

RS.lemmas . working.majority 

RS.majority .maj.exists 

RS.to.US.RSmap 

RS.to.US . reachable 

RS.to.US . reachable. in.n 

US . H.us 

US. initial.us 

absmod . abs 

clkmod. Corr 

clkmod. SI 

clkmod . SIC 

clkmod. S2 

clkmod. T.sup 

clkmod . enough.clocks 

clkmod . goodclock 

clkmod. in.R.interval 

clkmod. nonfaulty.clock 

clkmod . num.good. clocks 

clkmod. rt 

generic. FT. dep.agree 
generic. FT. maj.condition 
multiplication. mult 
rcp.def b . distinct .phases 
rcp.def b . member.phases 
rcp.def s . next.phase 



rcp.def s . prev.phase 
sets[EXPR] .empty 

Total: 91 

The formulae used are: 

DA.br oadc as t.prf .brl 
DA.broadc as t.prf .brl a 
DA.br o adc as t _prf . br 2 
DA.broadc as t.prf .br3 
DA_broadcast_prf.br3.aa 
DA.broadc as t.prf . br4 
DA.broadcast.prf .br5 
DA.broadcast.prf .br6 
DA.broadcast.prf .br7 
DA.broadcast.prf .br8 
DA.broadcast.prf .br9 
DA.broadcast.prf . int 5 
DA.broadcast.prf .rtpO 
DA.broadcast.prf . rtpOa 
DA.broadcast.prf .rtpl 
DA.broadcast.prf .rtp2 
DA.broadcast.prf . rtp3 
DA.broadcast.prf .rtp4 
DA.broadcast.prf .rtp4a 
DA.broadcast.prf . rtp4b 
DA.broadcast.prf . rtp5 
DA.broadcast.prf . rtp6 
DA.broadcast.prf . rtp7 
DA.broadcast.prf _tcc .p_br8_TCCl 
DA.intervals .br.int 
DA. intervals . into 
DA. intervals . int 1 
DA.intervals . int la 
PA.intervals . int2 
DA.intervals . int2a 
DA.intervals . int3 
DA.intervals . int4 
DA_invariants.base.state.ind 
DA. invar iants . cdi_12a 
DA.invariants . clkval.inv 
DA. invar i ant s . clkval.inv.il 
DA.invariants . clkval_inv_12 
DA.invariants . cum.delta.inv 
DA.invariants . cum_delta.inv.il 
DA.invariants . cum_delta_inv_12 
DA.invariants . cum.delta_inv_14 
DA. invariants. da. rt.lem 
DA.invariants . enough.inv 
DA.invariants.enough.inv.il 
DA.invariants . enough_inv_12 
DA.invariants . ind.state.ind 
DA.invariants . Iclock.inv 
DA.invariants . lclock.inv.il 
DA.invariants . lclock_inv_12 
DA.invariants . lclock_inv_12b 
DA.invariants . Iclock.inv. 12c 
DA.invariants . Iclock.inv. 13 
DA.invariants . Iclock.inv. 14 



DA. invar ianta.nfclk.inv 
DA. invar i ant s .nf clk.inv.l 1 
DA.invariants . nf clk_inv_12 
DA.invariants . rtl 1 
DA.invariants. state. induct ion 
DA. invariant s.t cc .Corr.lem.TCCl 
DA.lemmas .ELT 
DA.lemaas . com.br oadcast.l 
DA.lemmas . coa_broadcast_2 
DA. lemmas . com.br oadcast .3 
DA_ lemmas .com.br oadcast .4 
DA.lemmaa . c oa.br oadcast_5 
DA. lemmas . coa.sync. 1 
DA. lemmas .com_sync_2 
DA. lemmas . com_sync_3 
DA.lemmaa . coa_sync_4 
DA.lemaas . com.vote. 1 
DA.lemmaa .com. vote_2 
DA.lemmaa . coa.vote.3 
DA.lemmaa . com.vote.4 
DA.lemaas . earliest. later.time 
DA.lemmaa .elt.a 
DA.lemaas. map. 1 
DA.lemmaa .map. 2 
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